What event upset Brian so much in Hatchet?

In Gary Paulsen's novel Hatchet, the main character, Brian, crash-lands into the wilderness in a small plane. Only two people are in the plane--just him and the pilot--and the pilot dies. Brian is left to fend for himself and hope to get rescued.

Although his time and thoughts are largely occupied with finding food and shelter, Brian is also bothered by thoughts about something that he calls the Secret. He thinks of it often and even has dreams about it. 

Readers don't find out exactly what it is until later in the story, but here it is: Before Brian got on the plane, he had seen his mother kissing a stranger. And it was this fact that his mom was cheating on his dad that caused their divorce.

Here's all we know about it in Chapter 1, while Brian is still in the plane:

"What he knew and had not told anybody, what he knew about his mother that had caused the divorce, what he knew, what he knew—the Secret."

The Secret gets revealed slowly to readers as Brian keeps thinking about it, and we get all the awful details in Chapter 7:

"Then he crawled back into the shelter and fell again to the sand but could not sleep at first, could do nothing except lie there, and his mind decided then to bring the memory up again. In the mall. Every detail. His mother sitting in the station wagon with the man. And she had leaned across and kissed him, kissed the man with the short blond hair, and it was not a friendly peck, but a kiss. A kiss where she turned her head over at an angle and put her mouth against the mouth of the blond man who was not his father and kissed, mouth to mouth, and then brought her hand up to touch his cheek, his forehead, while they were kissing. And Brian saw it. Saw this thing that his mother did with the blond man. Saw the kiss that became the Secret that his father still did not know about, know all about."

Of course, this bothers Brian immensely. He held the Secret inside and didn't tell his mom or his dad; he worried that what he'd seen had caused the divorce, and it made him feel ill and distressed.