A difference system of sets (DSS) is a collection of subsets of , the integers mod , with the property that each non-zero element of appears at least once as the difference of elements from different sets. If there is just one set, it is called a principal DSS. DSS arise naturally in the study of systematic synchronizable codes and are studied mostly over finite fields when is a prime power. Using only triangular numbers mod , we constructed a DSS over for each positive integer . Necessary and sufficient conditions are given for the existence of a principal DSS using only triangular numbers in terms of coverings of by finite arithmetic progressions.