Steppe Lemmings need a large tank for their environment with a meshed lid for ventilation. They are unable to jump and are a poor judge of height so should not have any steep inclines or ledges. Providing you are confident that they cannot pile their substrate high enough to climb out, you may not need a lid for your tank.
Since they are avid burrowers in the wild, in captivity it is advisable to give a thick layer of a carefully chosen substrate. Steppe Lemmings are prone to respiratory ailments so it is important to not use anything sawdust, pine or cedar based and ensure that their bedding or substrate does not become damp. Water bottles should be monitored carefully for leaks.
A layer of hay makes both good concealment and also provides an essential part of their diet since in the wild they would feed on dried grasses. Like all rodents, Steppe Lemming have teeth that are always growing so some branches or twigs should be provided for gnawing material. Willow branches are a good choice for this.
A solid based wheel is ideal to help them burn off their energy, as they seem to have periods of rest followed by heavy activity. You can also provide plenty of cardboard for chewing and various tubes, wooden ledges and nests.
An important thing for Steppe Lemmings, especially if using only one nest is that it must have more than one entrance hole. This is due to the attack style of the lemmings and allows a bullied lemming to escape its aggressor more readily.