Ki Sung-yueng and Gylfi Sigurdsson found the net either side of a Wayne Rooney scissors-kick to give the Swans a surprise 2-1 win.
The new United manager pulled no punches after the final whistle.
The Dutchman criticised his players for being nervous, lacking confidence, and failing to play as a team. In fact, the 63-year-old said his first official game in charge of the 20-time English champions was the stuff of nightmares.
Van Gaal swept into Old Trafford with a swagger after United racked up six straight wins against some of the best teams in Europe in pre-season. They got a reality check in Van Gaal's first Premier League game.
"When you have preparation time and you win everything and your first match you lose, it cannot be worse, I believe," Van Gaal said. "It's a very bad day for us. It's very disappointing for the players, the fans, for me, and the staff."
United lacked a cutting edge up front and their passing was sloppy at times. But the main concern for Van Gaal will be in defence. Both Sigurdsson and Ki were unmarked when they beat David De Gea.
There was a clear lack of organisation at times at the back, with Phil Jones and Chris Smalling both looking out of their comfort zone on several occasions thanks to Swansea's enterprising play and neat passing.
United were crying out for the experienced heads they lost this summer.
The defeat to Swansea, among the favourites to go down, suggests that Van Gaal desperately needs defensive reinforcements, and the quicker he moves in the transfer market, the better.
The toughest task of Van Gaal's reign so far will be picking his men up again for next week's trip to Sunderland.
"We have built up a lot of confidence and then it shall be smashed down because of this result," Van Gaal said.
"We are not reaching our level. That's the main thing. Because of that we didn't play as a team."
The United manager thinks many of his players buckled under the pressure of trying to make a positive start to the season after the dire showing last year under David Moyes.
"I saw a lot of players looking very nervous, making the wrong choices and that is a pity," the former Barcelona and Bayern Munich coach said.
When asked if he needed to add new players to his squad, Van Gaal said: "Yes, but I thought that before this match. I know in what positions we need better players. But you have to buy only when the player can fulfil the way of playing that I ask."
One plus point for Van Gaal is that Robin van Persie is close to full fitness following his late return from the World Cup.
Garry Monk was delighted to have led Swansea to their first league win at Old Trafford.
Monk thinks his team can go on to surprise a few more people.
"Nobody gave us a chance before the game, but we can be better than that," the Swansea manager said.
"We can be sharper and improve in certain areas, but today we were right in many aspects, and we got the right result.
"I'm very pleased overall. We were very good today and showed a great attitude. Hopefully we can have more days like this."