Just by watching, it relies on the camera's fixed position (and it's single point perspective). Even when they move it around they're careful to keep it lined up correctly oriented to the camera's perspective and line of sight. At the end of the video you can see that the wooden frame is not a rectangle with 90 degree corners (note how the proximal end of the frame, relative to the man, is not parallel to the distal end of the frame). This would lead one to suspect a spacial illusion involving a forced (false) perspective. At this point, though not solved, you know an illusion is involved, at which point I begin to suspect all factors they don't rule out ahead of time, i.e., the actual plane of the table top and even the orientation of the actor (one assumes he's sitting in a normal chair but he could be "standing in a hole" at the optimum height for the illusion).
If it relied on an illusion of one point perspective then all the action should take place at the center. If you note in the video, all the tiles are slid to the center each time, and then the build begins from that "new" center.