Those most tempted by takeaways and fast food eat an extra 40g (1.4ozs) of high calorie food - the same as half a small serving of McDonald's fries - every week compared to those who stay away. Over a year the total eaten would be 2kg (4.5lbs).
Working near a fast food place or takeaway caused the biggest problems, closely followed by them being located near the home.
The research, published online in the British Medical Journal, is the first UK study to combine data from home, work and commuting and involved 5442 adults aged 29 -62.
On average, people were exposed to 32 takeaway outlets - nine each in their neighbourhood and on their commute, and 14 within a mile of work. There were about 48% more takeaway outlets and fast food joints near work compared to home, the study found.
Researchers examined how much takeaway food people ate and measured people's body mass index (BMI) as a measure of their weight.
The results showed those exposed to the highest number of takeaways were 80% more likely to be obese and 20% more likely to have a higher BMI than those with the lowest number of encounters. They also ate more of these types of foods.
Dr Thomas Burgoine, lead author of the study, said: "The foods we eat away from home tend to be less healthy than the meals we prepare ourselves, so it is important to consider how exposure to food outlets selling these high calorie foods in our day-to-day environments might be influencing consumption.
"Our study provides new evidence that there is some kind of relationship between the number of takeaway food outlets we encounter, our consumption of these foods, and how much we weigh.
"Our findings suggest taking steps to restrict takeaway outlets in our towns and cities, particularly around workplaces, may be one way of positively influencing our diet and health."