Intellectualization of agricultural machinery can effectively improve the efficiency and quality of operations, and has an important role in promoting agricultural development. Based on AR technology, this paper introduces the key technology to build the interactive control system of agricultural machinery, uses NURBS to realize virtual agricultural machinery modeling, uses VRML technology to design a prototype of the scene environment of interactive farmland virtual reality, and details the methods of virtual modeling, virtual roaming, interactive control and collision detection in the process of system development. A four-degree-of-freedom simulation test bed is established to realize the simulation of the tractor’s attitude when walking in the field. The position information of the crop rows is extracted from the virtual scene, and the control signals are given according to this information to carry out the speed, direction and balance control of the traveling of the agricultural machine, so that the tractor travels along the crop rows. The maximum deviations of the roll angle, pitch angle and yaw angle are within 0.36°, and the maximum deviations of the elevation and traveling speed are 2.11 mm and 0.14 km/h. The simulation analysis and the physical test show the feasibility of the interactive control system of the farm machine.