I could buy that they decided to leave on camera, IF they told Rick they were leaving! But, they haven't. In fact when they got back to their place, the sitter was there and Danny said "See you next week". SO, it didn't sound like they were leaving. If they had already chosen to leave, he would have told the sitter she was no longer needed.

At least we know why Robbie was not there. He was sick. At least they explained WHY. Hopefully we can see the kids tomorrow one last time?
Nancy