A demo showing the Xbox One swapping seamlessly between types of content.

How the Xbox One draws more processing power from cloud computing RB also talks to Microsoft about the integration of Kinect and RAM allocation.

While Tuesday's Xbox One presentation answered some questions about Microsoft's upcoming system, it left just as many or more unsettled. Luckily, Ars got a chance to sit down with General Manager of Redmond Game Studios and Platforms Matt Booty to try to get more answers. While he wasn't able to answer some of the most pressing questions about the system, he was able to dive deep into some of the technical details.

Our first question had to do with the 300,000-server cloud architecture that Microsoft says the Xbox One will use to help support "latency-insensitive computation" in its games. What does that mean exactly, and can laggy cloud data really help in a video game where most things have to be able to respond locally and immediately?

"Things that I would call latency-sensitive would be reactions to animations in a shooter, reactions to hits and shots in a racing game, reactions to collisions," Booty told Ars. "Those things you need to have happen immediately and on frame and in sync with your controller. There are some things in a video game world, though, that don't necessarily need to be updated every frame or don't change that much in reaction to what's going on."

"One example of that might be lighting," he continued. "Let’s say you’re looking at a forest scene and you need to calculate the light coming through the trees, or you’re going through a battlefield and have very dense volumetric fog that’s hugging the terrain. Those things often involve some complicated up-front calculations when you enter that world, but they don’t necessarily have to be updated every frame. Those are perfect candidates for the console to offload that to the cloud—the cloud can do the heavy lifting, because you’ve got the ability to throw multiple devices at the problem in the cloud."

Booty added that things like physics modeling, fluid dynamics, and cloth motion were all prime examples of effects that require a lot of up-front computation that could be handled in the cloud without adding any lag to the actual gameplay. And the server resources Microsoft is putting toward these calculations will be much greater than a local Xbox One could handle on its own. "A rule of thumb we like to use is that [for] every Xbox One available in your living room we’ll have three of those devices in the cloud available," he said.

While cloud computation data doesn't have to be updated and synced with every frame of game data, developers are still going to have to manage the timing and flow of this cloud computing to avoid noticeable changes in graphic quality, Booty said. “Without getting too into the weeds, think about a lighting technique like ambient occlusion that gives you all the cracks and crevices and shadows that happen not just from direct light. There are a number of calculations that have to be done up front, and as the camera moves the effect will change. So when you walk into a room, it might be that for the first second or two the fidelity of the lighting is done by the console, but then, as the cloud catches up with that, the data comes back down to the console and you have incredibly realistic lighting."

Does that mean that Xbox One games will feature graphics that suddenly get much more realistic as complex data finally finishes downloading from the cloud? "Game developers have always had to wrestle with levels of detail... Managing where and when you show details is part of the art of games," Booty said. "One of the exciting challenges going forward is a whole new set of techniques to manage what is going to be offloaded to the cloud and what’s going to come back.”

And what about those times when a gamer doesn't have an active Internet connection to make use of the cloud's computational power? Microsoft has confirmed that single-player games don't have to be online to work, but all this talk of cloud computing seems to suggest that these games might not look or perform as well if they don't have access to a high-speed connection.

"If there’s a fast connection and if the cloud is available and if the scene allows it, you’re obviously going to capitalize on that," Booty told Ars. "In the event of a drop out—and we all know that the Internet can occasionally drop out, and I am saying occasionally because these days it seems we depend on the Internet as much as we depend on electricity—the game is going to have to intelligently handle that." Booty urged us to "stay tuned" for more on precisely how that intelligent handling would work, stressing that "it’s new technology and a new frontier for game design, and we’re going to see that evolve the way we’ve seen other technology evolve."

Requiring the Kinect

On Tuesday, Microsoft announced that a new Kinect sensor would be included in the box with every Xbox One and subsequently said that the device has to be plugged in for the system to function. My first reaction to this news was that it seemed like an unnecessary cost being forced on gamers who might not want to invest in motion controls or voice commands in order to play standard games on the Xbox One.

Booty had a different perspective on the Kinect requirement, though. "From a design point of view, we don't see [the Kinect] as something that would be separate from Xbox One itself." Booty implied that the Kinect could have even been built into the Xbox One casing, if not for concerns about players wanting the Kinect and the Xbox One in slightly different places in their living rooms. "It just happens to be in a different plastic housing, and I think that's just because of location."

To Microsoft, the Kinect is just as crucial to the Xbox One experience as some truly basic input options are to the modern PC experience. "It’s kind of like going back to the days before PCs had mice, and games would have to say 'this game supports a mouse.' You would never think that any computer these days would not have a mouse or some sort of touchpad. I think that’s the way we want to think about Kinect going forward. ... A laptop would be cheaper without a keyboard, but you wouldn’t think of doing it without it, right?”

Those who have moved on to smartphones and tablet computing might quibble with Booty's analogy there. But Booty also stressed that integrating the Kinect with the Xbox One is about more than just skeletal tracking and motion-based gaming. "It is about personalization, it is about recognizing your voice and recognizing you, it is about being able to empower and enable communications through things like Skype. We really see it as a multi-purpose sensor in the living room that’s perfectly suited to an all-in-one entertainment box."

On the gaming side, Booty argued that packing a depth-sensing camera in with the system could help get controller-free, full-motion gaming out of the small niche it currently occupies. "Because it is known to always be there, because it is a fixed performance spec, because it is something you can depend on, I think developers are going to find all kinds of new things to do with it because they don’t have to worry about it being a unique accessory or something that only a small portion of the user base has," he said.

Other tidbits

The 8GB of RAM in the Xbox One is accessible as one unified chunk and can be divvied up between the GPU and the CPU dynamically. "Different kinds of games will want to use the CPU, the GPU, and the RAM differently, Booty said. "If we were to try to determine ahead of time how you wanted to divide RAM up between video RAM, main RAM, and audio RAM, we’re making a lot of assumptions, and when you make assumptions you start to limit the people who are going to be making games going forward."
While the 500GB hard drive built into the Xbox One won't be easily user accessible or swappable for a larger unit, players will be able to add more storage through external USB drives, Booty said.
The Xbox One will make use of an IR blaster to change channels on various TV, cable, and satellite systems.
The system will also have an external power brick to make it easier for a single design to deal with power needs from different countries, though Booty didn't have any details on its size or design.
Booty dodged a question about whether or not the Xbox One will be able to record TV shows directly, as it does gameplay, but hinted that the feature would be unavailable due to rights issues.
Booty also dodged the important question of how used games and game installs will actually work on the system. "Stay tuned for the details, but we do want to support secondary use of games," is all he would say on the issue.

Xbox One: not always on, not backwards compatible

Two big questions are addressed definitively, but some remain about secondhand sales.

Microsoft has at long last provided confirmation that its next generation console, the Xbox One, will not require an always-on Internet connection.

The company says that it will require an Internet connection at least some of the time, however. The console will use its "mostly on" Internet connection to perform system management tasks such as downloading updates in the background, backing up game saves to cloud storage, and synchronizing things like the TV shows or movies that you're watching between consoles so that you can start watching a film on one device and pick up where you left off on another.

Games will also be required to be installed to the hard disk and won't need optical media to play.

This has implications for second hand games. Games will be tied in some way to Live IDs. If you install the game with a second Live ID, there will be some facility to pay money and use the game with that second Live ID; both IDs will then have full access to the game without needing the optical media.

That's fine for concurrent users but isn't appropriate for second hand sales, where the original owner forfeits their right to use the game and the new owner acquires that right. Microsoft says that it has designed Xbox One to "enable customers to trade in and resell games" but won't say any more on this subject or how that will work until later in the year.

Update: Well perhaps things aren't as definitive as they seem. Wired, which had extensive early access to Microsoft and the Xbox One, states unambiguously that both people will have access to the game ("the new account would also own the game"). One of the official Xbox support accounts on Twitter, however, is saying explicitly that there will be no fees. But with Microsoft still not explaining how secondhand sales will actually work, it's difficult to know if both Wired and the Xbox support are talking about the same thing.

Update 2: It looks like they were indeed talking about different things. Kotaku is reporting that multiple people will be able to install the game from one disc, with the second and subsequent installers paying online to activate their copy. This price will be consistent with the then-current retail price. Kataku's information came from an interview with Microsoft Corporate Vice President Phil Harrison. This confirms what Wired wrote and what we originally reported. However, in addition to this, there will be some as-yet unspecified system for performing license transfers, which is to say, secondhand sales.

The new console also won't be backwards compatible with the Xbox 360. This is entirely unsurprising given the change in processor architecture. Software emulation of the Xbox 360's three core, six thread, 3.2 GHz PowerPC processor on an eight core, x86 processor expected to run at around 2GHz is technically unfeasible.

Microsoft talks about Xbox One’s internals, while disclosing nothing

Eight cores, six operations per CPU cycle, and 200GB of bandwidth. But does it mean anything?

Here's the money shot: the back of the console has a power connector, HDMI in and out (for the purposes of hooking your cable box up to the console), optical audio out, two USB ports, the Kinect port, an IR Out port, and an Ethernet jack.

The Xbox One is full of technology, and after its big reveal Microsoft talked a little about what's going into the console, giving some tidbits of info about what makes it tick.

Hardware

Microsoft says that the Xbox One has five custom-designed pieces of silicon spread between the console and its Kinect sensor. It didn't elaborate on what these are. There's a system-on-chip combining the CPU and GPU, which we presume to be a single piece of silicon, and there's at least one sensor chip in the Kinect, perhaps replacing the PrimeSense processor used in the Xbox 360 Kinect, but what the others might be isn't immediately clear. Possibilities include audio processors, on-chip memory, and USB controllers.

One of the key questions about the AMD-built, 64-bit, 8-core SoC is "how fast is it?" At the moment that's unknown. Microsoft claims that the new console has "eight times" the graphics power of the old one, though some aspects of the new system are even more improved; for example, it has 16 times the amount of RAM.

The SoC has a PC processor heritage. It includes features that have become standard in PC processors, like power gating to allow idle cores to be powered down, and dynamic frequency scaling to allow light loads to use a lower clock speed. Like AMD's forthcoming codename Kaveri processors that are shipping in PCs later this year, the CPU and GPU share coherent access to the system's memory, making it easier to develop software that splits workloads between the two processors.

Some performance numbers were given for the CPU and GPU themselves but these cast more shadow than they do light. Microsoft claimed that each CPU core can perform six operations per cycle. The CPU is believed to be using AMD's Jaguar core, but typically this would only be described as able to handle four operations per cycle; two each of integer and floating point (though even here counting operations is complicated; the floating point operations could use vector instructions such as SSE2, in which case one operation would result in four actual computations, potentially giving eight per cycle for floating point alone).

This arguably leaves a shortfall of two operations per cycle. One possibility is that the cores have been customized somewhat, which allows more instructions to be issued per cycle. On the face of it, this seems a little unlikely; it'd be a significant change that would have considerable implications on the design of the rest of the chip. Another possibility is simply that the counting is a little unusual and that the extra two operations are one store and one load. This would be consistent with how leaked documents (or, if one prefers, unsubstantiated but apparently accurate rumors) described the processor: two integer operations, two floating point operations, and two memory operations per cycle, which may well be the same number and mix of operations as handled by the standard Jaguar core.

For the GPU, Microsoft claimed 768 operations per cycle. This is again consistent with leaked information.

What Microsoft didn't specify, of course, was the number of cycles per second each processor runs at, so we still have no basis for actually assessing the device's performance.

Similarly, the company claimed that there was more than 200GB of bandwidth within the system. Again, the number had no context or clarification and if rumors are to be believed, it suggests some rather creative accounting: 68 GB main memory bandwidth, 102GB bandwidth to an embedded SRAM buffer for the GPU, and 30GB bandwidth between the CPU and GPU. While that does add up to 200GB, there are no two parts of the SoC that can communicate with each other at 200 GB/s. The fastest link is believed to be the GPU read performance, which can aggregate across the main memory and SRAM buffer for 170 GB total.

The Kinect system has also been upgraded. Perhaps most importantly of all, it should work a lot better in small rooms. The field of view is described as being 60 percent wider and this translates to being able to stand 3-4 feet closer to the sensor. That's a substantial improvement, which is just as well since the Kinect will be mandatory equipment.

The new Kinect is all around better. It can track six skeletons, up from 2, and capture 1080p video. Low-light performance will also be improved, as it can see infrared. Microsoft says that this will allow Kinect to gauge things like your level of engagement in a game. We speculate this means that it might be able to see, for example, that your cheeks are flushed in response to emotional involvement and investment in the game.

Software

Brief details were also given of the software side of things. The Xbox One is described as running three operating systems. There's a long-running Windows 8-based operating system used for running applications, browsing the Web, Skype, and similar roles; a second operating system for running games; and a hypervisor that virtualizes the hardware and switches between the two. The long-running partition is also used for some system management tasks, such as running the Kinect software portions and performing matchmaking while you watch a video.

This hypervisor is based on Microsoft's Hyper-V Virtualization platform but simplified to remove extraneous features that are irrelevant to a games console; the software is specialized because it runs fixed-role, fixed operating system virtual machines.

The application partition boots when the system is turned on and runs persistently, even when in-game. This is what enables things like Snap view, where apps and games or TV run side-by-side. The game partition, in contrast, gets rebooted each time a new game is started.

To ensure high-speed switching between the operating systems, each virtual machine draws to its own (virtual) screen all the time. The hypervisor can switch between screens essentially instantly, allowing fast task-switching.

The mystery box

Microsoft claims that the session gave a lot of detail about the new console. And in some ways, it did. There were lots of isolated pieces of information, without much context. The company simultaneously told us something about the Xbox One's performance and told us nothing. It tossed out some numbers for people to talk and argue about (because God knows the console platform warriors certainly need more ammunition) while ensuring that those same numbers mean absolutely nothing. The Xbox One may have been revealed, but many facets of its capabilities and power remain mysterious.

Hands-on (and on camera) with Xbox One’s new controller and Kinect

Rumbling triggers, infrared night-vision, and a creepy visual heartbeat monitor.

REDMOND, WA—The schedule for today's Xbox event on Microsoft's Redmond campus included time for what was billed as a half-hour "interactive experience." That experience unfortunately did not include a chance to sample any actual gameplay running on Xbox One hardware, but it did include an opportunity to get some time with the new Xbox One handheld controller and revamped Kinect camera.

A group of journalists and I were first taken to one of Microsoft's testing labs to check out the new controller. The main point of this demo was to show off what Microsoft is calling impulse triggers, a term that refers to magnetic motors in both the left and right triggers that add a highly variable rumble directly to your fingertips (which are much more sensitive than the palms of your hands). This is in addition to two more traditional and more powerful rumble motors in both sides of the base of the controller.

These new triggers were shown off in six simple demos that coordinated force feedback with animation on the screen. The demos encompassed an impressive range of rumbling power for the triggers, from the subtle lub-dub of a heartbeat and a gentle laser gun shot to the rat-a-tat of helicopter blades and a powerful car ignition.

It's an odd sensation at first, feeling more like a gentle tickle than the real, powerful force feedback most gamers are used to. The light touch works though, adding a directionality and a level of fine-tuned impact that is lacking from standard controller rumbles. The best way I can describe it is as the touch equivalent of one of those surround sound stereo system demos that adds a sense of motion to sound that once seemed flat and unidirectional.

The most intriguing impulse trigger demo was probably one that combined the more traditional force feedback motors to simulate the rumble of a car engine, then used the impulse triggers to highlight the squealing of brief taps on the brakes. I could see this acting as an important new gameplay cue, outside of sight and sound, that could help add a new level of intuitive response to racing games that's not possible on other systems.

The actual controllers were early prototypes and not final designs, so it was hard to gauge other new controller features like more textured grips on the analog sticks (which we're told are smaller and require 25 percent less force to move than the Xbox 360 controller). The controller did feel more comfortable in my hands, thanks to the lack of a jutting battery pack in the back that can bump against the fingers on the Xbox 360 controller.

The new, plus-shaped d-pad felt perfectly fine on my fingers, although it's hard to gauge for sure how well it works until they let us loose on some fireball motions on an old-school 2D fighter. Microsoft pointed out later that the new controller also has repositioned the seams and screw holes that annoy the fingertips of some Xbox 360 users, but I can't say I really noticed at the time.

A better Kinect

From there it was on to a demonstration of the new Xbox One Kinect sensor. From the moment we entered the demo room, it was clear how much the new sensor improved on the depth sensing resolution of the old Kinect, just from viewing a live, greyscale 3D model of a couch sitting about seven feet from the sensor (a representative said it was about three times the fidelity of the old Kinect). When a volunteer got up in front of the couch, I could easily make out details, from the ripple of his shirt as it fell on his chest to the individual fingers as he rotated his wrists.

The new Kinect doesn't have the motor that the old unit used to make sure it was pointed perfectly at the room in front of it. Instead, the unmoving slab simply has a much wider field of view that captures substantially more of the room when placed on a level surface. A six-foot tall volunteer was completely visible to the sensor while standing only three or so feet away, based on my rough eyeballing of distances.

The demo then switched to a 1080p live color feed of the room, which was unimpressive on its own to anyone with a decent webcam. What was more impressive was the Active Infrared feed, which showed a night-vision style greyscale view of the room at high resolution, even when the room was pitch black (or only lit by a flashlight). This should help for things like computer vision algorithms, which have previously needed extremely bright and consistent lighting conditions to detect certain details in the flat images provided by the Kinect's RGB camera (and yes, the depth-sensing works in the dark as well).

After a quick demo of how the new Kinect's four-microphone array can filter out distracting sounds from a TV speaker, I jumped in front of the camera to test how the new Kinect's improved skeletal tracking worked. The first thing I noticed was a complete absence of the kind of lag and jumpiness that so often characterized avatars on the original Kinect. The second thing I noticed was the way the on-screen skeleton could detect even small motions like a rotating wrist or a shrugging shoulder.

When the avatar in the demo changed from a skeleton to a thicker person made of blocks, it showed how the new Kinect could easily detect which direction I was looking as I tilted my head at the neck. It can also measure the amount of force being put on either side of the body, Wii Balance Board style, and project the force of a punch or a kick based on the speed of the fist behind it.

The creepiest part of the Kinect demo was when the system used a combination of the RGB and IR cameras to detect my pulse rate just by looking at my face. Not only that, but the system could tell when I was smiling and/or looking away from the screen and tell which of two controllers I was holding, even as I handed one off to the demonstrator (a process the PR rep said worked by "magic").

It's still hard to say whether the new Kinect is truly responsive and high-resolution enough to fulfill the promise of controller-free, motion-based games. Still, the improvements to the raw specs, combined with the fact that literally every Xbox One owner will have a Kinect right out of the box, have us tentatively excited about the potential that developers will be able to squeeze from it.

RBI

Pages

Saturday, May 25, 2013

Microsoft announces its next console, the Xbox One It's "the ultimate, all-in-one home entertainment system."