Windows Delivery Optimization – the 2Pint Edge

In our many customer engagements over the years, there’s a topic that I can guarantee will come up in the initial ‘discovery phase’ conversations. That topic is ‘Good old DO’ as we call it here, or to give it its proper title, ‘Windows Delivery Optimization’. (henceforth known as DO)

The question we ask usually goes something like this. ‘So how are you managing DO today, and what workloads does it manage in your org?’ Then there are broadly speaking two types of response from the customer:

1.       We use GPO/MDM/ConfigMgr (sometimes all 3) to manage DO, and it’s only really used for Windows Store Apps/Updates

2.       We don’t use DO..

So, the purpose of this article is to expand on both of these scenarios and to attempt to dispel some of the myths and untruths around DO – specifically WHAT is it downloading and HOW should/could we manage that.

Scenario 1 - ‘Of course we manage DO!’

For those folks who have at least entered the minefield of DO management, there seems to be a multitude of approaches, ranging from ‘oh yeah this guy from Microsoft came in and set it all up for us, but not sure what he did..’ to ‘Yes! We use the settings in ConfigMgr – that’s should be fine right!?’  Of course there are many other answers in between but you can start to see the problem here..

Scenario 2 – ‘Nah mate, we turned that DO off, cos we don’t really use it’

 This is actually the easiest to deal with because you are starting with a blank canvas here. Ignorance is bliss as the saying goes.There’s a great TechNet (remember that?) article that demonstrates the frustration here quite well..

https://social.technet.microsoft.com/Forums/en-US/cec4880d-a440-448d-9749-64c29f87726c/windows-10-quotdelivery-optimizationquot-off-and-still-consuming-recources-and-bandwidth?forum=win10itprogeneral

but really, the bottom line is you can turn DO off but you can’t turn it ‘properly off’ – nor should you want to! And part of understanding why this is a bad idea is in understanding just WHAT DO needs to download.. but more of that later.

OK 2Pint smartasses – what’s the problem here?

Thanks for asking 😊

Part of the problem is that Microsoft themselves don’t really do a great job in communicating both HOW DO works and WHY it needs to be properly managed. There is plenty of information out there on how to set DO policies, how to use Intune/ConfigMgr to manage DO – but none of these discrete chunks of info actually give you the bigger picture. So here it is.

The bigger picture

DO has become a beast to manage. As of the Windows 11 release, DO now boasts a total of 30 YES 30! GPO/MDM settings. These settings control the following areas of interest when we are downloading content:

Network Impact

Cache performance

P2P Behaviour

Scheduling

System Resource usage

So while it’s great that we have this potential for granularity of control, what we also have is an ever growing matrix of settings for each DO scenario. Even Microsoft have a set of guidelines here which more or less admit as much. What this set of guidelines says is..

Do you have:

Hybrid WAN? Hub and Spoke with Boundary Groups? Lotsa mobile devices? Plenty of free disk space and large numbers of devices?

It then goes on to give you a set of settings for each scenario. BUT! You can add a few more questions to that list such as:

Wired or wireless? Fast or slow networks? Users that move between locations? Home workers? VPN? Proxies? 3^rd Party Firewalls? Wonky networks where P2P is disabled? Hmm Microsoft seem to have forgotten to mention those, odd. Oh, and what happens when your users move around? They do that sometimes don’t they – annoying.

Are you starting to see the light yet? 😊

OK OK I get it!, but why should I care? I mean we mostly use ConfigMgr so DO isn’t really doing anything is it?

Well. Sorry to urinate on your bonfire of delusions my friend but.. yes, it is. Remember we mentioned earlier that we needed to understand why it’s a bad idea to try to turn DO off? Here goes, hold on to your pants!

To start with, here is a list of content that DO supports:

I mean that’s a lot of content, right? But that’s not all! You can tack onto that list 3^rd Party vendor updates, which can add up to a fair old chunk ‘o’ bytes.

And in that list, the seemingly innocuous item named ‘Windows 10 Store Files’ hides that fact that ‘Store Files’ confusingly includes  all of the updates to a TON of builtin Windows components such as Language Packs, Calculator, Photos, C++ runtimes, Help, Maps (the list goes on and on) heck, even the Windows login screen is a store app.

Still want to turn off DO? Because if you do somehow manage to do that you are effectively blocking updates to all of the above components. Annd we know how that can end. Don’t be that person.

I mean, even if you did want to ‘stop using DO’ by setting the Download mode to 100, which bypasses DO and uses BITS instead, all you are doing is ensuring that there will be zero Peering of content. That is a bad thing generally. Oh, and Bypass mode is deprecated in Windows 11 anyway. So there.

I mean you could just disable the DO service yes? Yes! We have seen that done too by some clever folks, but actually all that does is force Windows to use BITS instead. D’oh.

ConfigMgr! I use SCCM so why do I need to bother with DO?

Yes, yes, we know that you love SCCM and have been using it since you were 5 years old and it is DEFINITELY NOT DEAD, but even if you are using ConfigMgr for the majority of your App/Update installs, DO is still chugging away in the background slurping some or all of the above content, all day every day. And if you drank the Microsoft koolaid that said that you can use DO in ConfigMgr via Express Updates, know this. In the background it’s so damn complicated it warrants a whole blog of its own. Spoiler: It mostly uses BITS, some DO for peering, and you might end up with content in your DO cache as well as your BranchCache cache. Oops.

Uploads. Huh?

Yeah, it ain’t just downloads buddy, all of that delicious telemetry? You guessed it – DO handles the uploads too.

How much!!?

We didn’t quite believe all of this ourselves until we did some analysis at a customer using our StifleR toolset to snapshot 2 days of content download activity. We logged every BITS/BranchCache and DO download over 2 x 24hr periods and the results were, well, a bit nuts really. See below if you don't believe me - graphs don't lie!

This was a customer with 50k endpoints who uses ConfigMgr primarily, some Intune, but CM was doing all of the major lifting and shifting. Turns out that DO was gobbling up over 2TB of bandwidth per day, and BITS (ConfigMgr) was only consuming a measly 1.9TB. We have a winner.

But Microsoft Connected Cache can help surely?

Oh you had to go there didn’t you. MCC can indeed help with offloading DO content is some scenarios. Couple of issues here:

1 MCC is only available (currently) with ConfigMgr. So if you are in Intune/Non Microsoft MDM solution shop you will have to wait for the ‘standalone’ MCC version. Don’t hold your breath on that one – it’s been in Private Preview since Jan 2020.

2 Even with ConfigMgr there are some issues that we found if you have a few MCC servers, namely that COM doesn’t ‘round robin’ the list of servers, so you end up with the first servers in the list getting battered by high numbers of http connections. This is not an issue for you lucky 2Pint StifleR customers btw ;-)

OK I am suitably concerned now – what can I do?

Something. As opposed to nothing that is. DO is taking on more and more workloads as the relentless march to ‘Modern’ management continues, so the worst thing that you can possibly do is to ignore it. If you have a fairly simple network with lots of bandwidth, plenty of disk space on your clients then you can follow the Microsoft guidance and skip off into the sunset singing ‘the hills are alive’.

More likely your network/users/bandwidth availability is a bit more challenging in places, in which case you need to at the very least do some reading and testing. Here at 2Pint we have been testing DO in every scenario we can think of for about 8 years now (yes DO has been around since Windows 10), and as it is a SaaS based solution, it is constantly changing. As a result of our extensive testing, we now have a set of ‘absolute best practice’ settings that work in every scenario. From bandwidth limited remote sites, fast ‘Head Office’ campus locations, Wifi, VPN, you name it we have it nailed! That’s great, but of course you then have to apply those settings and keep them updated for ALL of your locations and user types. We don’t want to apply the same settings for a home user coming in over VPN as we would for a user sitting in a remote office on the end of a 1Mbit WAN do we?

Templates!

This is why we came up with the idea of having a template based approach in our StifleR product. We have created a template for every type of LAN/WAN/VPN scenario that we could think of, and these are dynamically applied depending on where the user connects from, so a user connecting from home will have a completely different set of DO settings to the user in the remote office, but if that user moves from their home office to the corp net, we make sure that the settings are changed accordingly. Neat huh?

Finally

Now obviously this is a blatant product plug for using StifleR to manage DO (I mean it does kick ass and we’d love to give you a demo) but you can also take control of DO using some basic strategies depending on your management solution. There are a ton of gotchas (looking at you overlapping boundaries in ConfigMgr) but there are also a ton of articles out there from the likes of 2Pint’s own Johan Arwidmark and even Microsoft if you want to grasp the DO nettle and wrestle it into submission. Or you can give us a call and we’ll just take all of that nasty complexity away and your Network team will love you for evermore.

If you want to know more you can catch us at MMS  in our Delivery Optimization Boot Camp session and we’ll give you the full soup-to-nuts version!

cheers, and happy peering :-)

Phil