Meeting with some Copilot+ PCs left me unimpressed. Why?

Its literally been a week since I walked into a Costco and took notice of the electronics area. This is the normal store layout: beautiful 8k and 4k OLED displays first followed by some quick-selling items (that day it was golf shirts), turn the slight corner and you have the laptop display right next to the tablets and phones. I normally skip this area quickly since I have a "phablet" (when was the last time you head that term used?!) in the trappings of a Samsung Galaxy Fold4 and a factory replaced Pixel 7 Pro (ever since the Pixel 6 Pro met its photographic hardware demise) and there is nothing in terms of a mobile carrier combo that the service people in that area can offer me better than what I, already, have.

However, something caught my eye that day. I must have been thinking about the projects I was doing and combination of hardware that are in this blog. Were those actual Copilot+ PCs on display? Not just one, but two!

Make and finish

I stepped closer and lifted each in turn, testing for the weight and sturdiness of how I might use it in day-to-day.

The keyboard on the Microsoft Surface Pro immediately dropped away. Ok! It felt almost exactly like the first time I held an iPad with its keyboard case more than a decade or so ago. Lighter. Plastic-like or flimsy is what came to mind. Somehow, my expectation due to the look was something akin to a Lenovo X1 Carbon – but, it wasn't. This thing will probably break in pieces if dropped on the floor together with that 16-hundred dollars and change.

The Asus Vivobook felt more familiar as a laptop. Well, actually, wouldn't it be more fair to compare this to a Chromebook? Unfortunately, even if it was plugged with lights underneath the keyboard no amount of fiddling with it could turn the display on. No audio or visual sensory feast here. It was simply "dead, Jim."

Out-of-the-box experience

When you put something on the floor display make sure it has every chance to impress. The only thing that seemed to have been done to the Surface Pro was make sure it was connected to the official demo Wi-Fi connection. Apart from that there was this rectangular widget-like window that I had to partially configure and accept EULA's in behalf of the next user after me. In truth, it doesn't seem like anyone had touched this thing since it rolled onto the floor if I had to do the initial set-up. After all the hype leading to and from the Microsoft event I was expecting fireworks, balloons, and cotton candy. If this was a State Fair in Texas then give me something fried and dipped in butter with a beer on the side! But, no. None of that.

Let's take it for a spin

There was no settings that I could configure (nor did I want to sign-in my own account). But, there should have been one set-up. Regardless, I was going to test it the usual way.

In the succeeding screenshots the fuzzy ones are from the Costco display and the clear ones are from Copilot in Windows (preview) on this desktop that I'm writing on.

This is a solo shot but if memory serves it was similar on the Surface Pro:

Now, let's ask the standard question:

How about some web search capabilities? Lets look for a blog on local AI usage from a local (this blog). This tests the location access rights the OS and app asked permissions for, as well:

As you can see it can't find the information that a simple Bing web search would:

And when somehow it tries to later on stay on topic locally for "Texas" it then fails in the overall topic context of technology and goes religion. Granted, such a link does exist if you've recently been watching some episodes of the 3 Body Problem. The standard non-Copilot+ PC has no issues with this question or context.

Here are my attempts to steer the Surface Pro Copilot+ PC to the right direction with clues that the LLM should be able to pick-up but, here, it continues to loose focus on the context and just spit out answers to the most recent question.

Honestly, my Amazon Alexa and army of Google Chromecast can do this better in spades and those aren't even upgraded to use Anthropic or Gemini, respectively!

Copilot with online strings attached

I mentioned early on that the WiFi was connected to the demo access point. While making sure I was getting the best experience via a normal speed test, I tried a few queries on this so-called Copilot app experience and was greeted with a need to be online for things to work.

What?! I mean, whatever happened to the point of having an NPU chip on this thing? How could a device that is 5-years newer be worse than what my home-built 12-core (24-threaded) AMD Ryzen desktop PC is able to do with the Copilot preview on the task bar?

I'm going to have to try sometime later, perhaps in Part 2, and install Ollama to force this Surface Pro to work as it should. That being said, it appears that is the same experience that PCWorld reported about the same time a week or so ago.

Bottom line: When offline, the current generation of Copilot+ PCs are dead in the water for anything AI related.

Game on

Let's not even go there? But, since you're here I might as well mention one more time that my AMD Raden 5700XT from 2019 gives me around 19.51 TFLOPS whereas the Surface Pro barely eeks out 4.6 TFLOPS. That's simply horrible for any serious gaming in terms of rendering framework and similarly what was, also, reported elsewhere. The fair but mismatched comparison of the Surface Pro would be against the same Qualcomm Adreno line-up performance of the Samsung Galaxy Fold4, but in a laptop/tablet-ish format.

Bottom line: No Cyberpunk 2077 for you!

Not blown away

It's ok. But, like I just showed, these demo floor units and the way Copilot+ PC (and AI PCs that are coming out from AMD and Intel later this year with more serious APU applications) need to think about the user experience if they're going to try to show the benefits of what generative AI can do.

Bottom line: Will the real Copilot+ PC with fireworks blazing please stand up? Please stand up?! Right now, this is meh.

Outtakes

This is where my testing ended. I didn't want to hog the demo units any longer than any casual buyer. However, as I stepped back to give the next person in line a chance to try is when I noticed there was no one wanting to do so.

Not one kid. Not one teenager. Not one curious geeky adult like me. What?

The Competition

In other news, Apple with the Apple Intelligence implementation which upgrades Siri on the iPhone 16 (partially backported to iPhone 15 Max) and MacOS Sequoia with OpenAI built-in comes out later this year in the Fall.

The way things are going I'm pretty sure Apple is going to go hard on AI functionality in every device they have so it is going to be interesting to see if demo lines are busier come that time.