Users would have little to no oversight over what is done with the data, which could be put in the hands of third-parties. Microsoft says Kinect 'cannot be removed' but it can be put into 'privacy mode'.
Don't want your new Xbox perving on you? Be sure to set it to deaf, dumb and blind mode. All manner of data is captured like voice, reaction rates, emotional state etc.
"The Xbox continuously records all sorts of personal information about me," Peter Schaar, Germany's federal commissioner for data protection and freedom of information. "Reaction rates, my learning or emotional states." They are then "processed on an external server, and possibly even passed on to third parties," he continued.
Schaar notes that the user cannot influence whether this data will ever be deleted.
Microsoft's Jeff Henshaw has stated that Kinect can be switched off: "The system is designed to have Kinect be an integral part of the experience. It's not the case where you'll be able to remove the camera altogether. But you'll be able to put the system in modes where you can be completely secure about the fact that the camera is off and can't see you."
Unlike with Xbox 360, Kinect is a mandatory requirement with Xbox One, where its latest design "understands the slightest rotation of wrist, shift balance, transfer of motion and when you're exercising it can read your heartbeat." Of course you could always just point the thing at a wall - take that software! Yeah I know you can hear me.
Xbox, Go Home.