In a tie of few testing shots, Watford's weakness in front of goal saw them have just one shot in normal time, a long-range effort from substitute Lee Cook which was well covered.
Their defence had looked solid enough until Lee Miller soared above them to connect with Aaron Brown's deep cross from the left in the 94th minute.
It was his second goal since joining the Robins from Falkirk for £300,000 in the summer.
Miller almost set up a second goal in the second period of extra time but Christian Roberts, after rounding Paul Robinson, shot against the post.
Both sides made three changes from Saturday's games, with the Hornets giving Ashley Young his first start of the season after being on the bench for all their other matches and the teenager had an eventful first half.
Inside 40 seconds he was fouled by Louis Carey to the left of goal, a good position for a free kick which City were relieved to scramble away.
Then, just short of the half-hour, he caught Carey, accidentally rather than maliciously, but he went into the book.
Gavin Mahon had been booked earlier for throwing the ball away at a free kick as Watford struggled to get a thin-looking attack moving.
They were not able to create an opening and after an early burst of high-paced action Watford fell away as City had more of the ball.
Brown set up two chances from the left, one which Miller headed just wide and another which Sean Dyche was forced to clear from Luke Wilkshire at the cost of a corner.
Watford's best chance fell to Bruce Dyer who went down under a challenge from Tony Butler.
Bristol City hotly disputed the free-kick on the edge of the area, but their anger faded when Paul Devlin's effort rebounded off the wall for a harmless corner.
The Nationwide Division Two side were on top and Watford used up their three substitutes inside the hour although one of them, Micah Hyde, lasted only 11 minutes before a hamstring injury troubled him again.