The herbalist skill is used in gathering plants from shrubs.
A highly skilled herbalist will not only pick shrubs more quickly, but will also be able to get a larger quantity per shrub (stacks); this has a positive effect when further processing and will allow for more efficient barrel use.
The herbalist skill will only affect shrubs that you did not plant as crops, and will only collect those you designate with the "gather plant" command.
Plant gathering can be a good food source early on, but tends to be rather useless later in game unless Kobold bulbs or Valley herbs grow in your embark region or you or feel the weird urge to do no farming or food trading at all.
Any plant appropriate to your surroundings can appear as a shrub. Once you discover the cave river, shrubs (and tower-caps) will start to grow on any muddy rough stone floors.
An herbalist is also responsible for processing kobold bulbs into Gnomeblight at a Still.