It ended a bleak spell of four months and 17 matches without a win in the National League South and was Borough's first clean sheet since September.
Gosport were in command from the moment Lee’s third-minute header from Tom Bird’s cross hit the back of the net.
Joe Lea made it 2-0 shortly after the half-hour and, just three minutes later, Southampton-based Lee struck again.
Sub Ollie Bailey bagged a fourth on 86 minutes and, although Borough remain five points adrift of safety in 20th, they will host Bath City on Tuesday with a renewed sense of belief.
Fifth-placed Poole Town battled out a helter-skelter 3-3 draw with play-off rivals Hampton & Richmond Borough.
Having led through Abdoulai Baggie and Luke Roberts, the Dolphins went from 2-0 up to 3-2 down before Marvin Brooks grabbed an 82nd-minute leveller, following up after James Granger’s free-kick had been parried.
The match marked the opening of new terracing at Tatnam and, asked if he was confident Poole would get the ground-grading to compete in the play-offs, vice-chairman Chris Reeves said: “I would find it inconceivable after everything we’ve done that we wouldn’t get it.”
An inspection team is due to visit on March 28.
On Tuesday Poole host the club directly above Gosport in the table, Truro City.