Calculations of the number of equivalence classes of Sudoku boards has to this point been done only with the aid of a computer, in part because of the unnecessarily large symmetry group used to form the classes. In particular, the relationship between relabeling symmetries and positional symmetries such as row/column swaps is complicated. In this paper, we focus first on the smaller Shidoku case and show first by computation and then by using connectivity properties of simple graphs that the usual symmetry group can in fact be reduced to various minimal subgroups that induce the same action. This is the first step in finding a similar reduction in the larger Sudoku case and for other variants of Sudoku.