The fortress guard is the military squad in charge of delivering justice within a fort. Headed by the captain of the guard, the fortress guard is otherwise completely identical to a standard military squad, and can be treated as such. However, since the fortress guard is the one to deliver beating (the second level of justice), it is a wise idea to keep your fortress guard incapable of too badly hurting an innocent dwarf whose only crime was carrying a good to the trading post that was forbidden for exports after the caravan had already left the fortress.
There are generally two ways to treat the fortress guard. The first is select a small number of unusually weak soldiers, give them crappy weaponry, not train them, and let them push around dwarves as they can, since they will be unable to deliver severe beatings - basically, you want the exact opposite of a proper military candidate. It's important to get the guards equipped, however, as unarmed dwarves will fall to choking and often strangling the poor criminals to death. The problem with this approach is that it makes your fortress guard an extremely crippled unit that would collapse when faced with a single ambush.
The second way is to make the fortress guard a marksdwarf squad, avoid cross-training them in the hammerdwarf combat skill, and equip them with extremely light crossbows. Fortress guards will never shoot criminals, instead bashing them with the crossbow; a ☼Feather Tree Crossbow☼ will not do much damage to a clothed dwarf in melee, and goes a long way towards mitigating beatings. This works well if you have a skilled bowyer, as high-quality wood or bone crossbows can still pincushion enemies with masterwork steel bolts, so long as you avoid getting them into melee combat. The best such crossbows are made of feather trees and adamantine (although you probably have better things to spend adamantine on).