USB & BranchCache – what sorcery is this?

With no blogs in almost a year, you might think things have died down here at 2Pint Software? On the contrary my dear Watson, we have been  too busy onboarding new customers and fiddling in our labs to blog about stuff. As we are now completing that work, it's time to start telling the stories we have been making. First out is a 'simple but complicated' piece of work we have done with USB drives for the 2Pint OSD Toolkit.

Problem background

There are some some scenarios where peer-to-peer (P2P) is not really useful or very efficient, and one of them is building a high volume of machines in a sequential fashion. If you build a low number of machines simultaneously, there are simply no peers to get to content from, as they have already been built, shut down, and shipped away. In order to solve this, we marry together the grandness of USB drivers with the awesomeness of BranchCache.

The plan

We mix fast USB keys with BranchCache, give it a stir, and back comes a hybrid method of building machines.

Speed is king

New USB 3.0/1 USB dongles have an amazing speed, up to 400Megabytes per second, which translates to about 3200 Megabit/s per machine. So that means, if you are building 5 machines in parallel, you can get 5 * 3200Mb/s  = 16Gigabit per second. The USB keys are much slower at writing, about 20Megabyte/s which translates to 160Mb/s. But as the write is a single operation, and all operations after that are read, it's definitely worth it.

As you can see from the Samsung marketing material, their 128GB models support up to 400Megabytes per second. The 32GB model supports half that, 200MB/s:

So this bad boy will not only speed up your downloads, it will also work if you happen to drop it into the toilet*! Epic!

*(Disclaimer: 2Pint in no way endorse the use of USB storage in the toilet )

Dynamic update

As BranchCache is block based, if data is updated on the server, the key will still have partial content available. Then during the build, things will update on the USB key automatically..

USB does not mean no Peer-to-peer

All the USB stuff really gives us is a local cache that is already populated. But if content is not present in BranchCache on the USB key, it will try to find another peer. As this CAN slow down the process, you also have the option to set volume name to "BranchCacheLocal", then the cache is 100% local and it will not try other peers. This is great if there are only a single machine being built at any time..

Secure

As the data that is stored on the USB stick is encrypted you can safely leave keys behind without caring too much..

Sounds great, how do I implement?

It's easy-peasy-lemon-squeezy (assuming that you are already down with the OSD Toolkit functionality that is):

Format a USB key with an NTFS partition and name it: "BranchCache" or "BranchCacheLocal"

BCEnabler.exe will then pick up the USB key if it's plugged in when the "Enable" step runs and then will use this for the WinPE stage. The BCEnabler.exe Move step will then move the BC cache onto to the OS disk.

The OS disk will then have a copy of the cache as it completes the apps/setup in the full OS. You can then run a sync via an upcoming tool that also allows iPXE and format the drives in the right way.

For USB based docking stations, this means you can leave the key in the station itself and voilà; you have an instant self updating cache ready for each new build faster than any ethernet. Noice!

Some things left to solve

The only thing left to sort out is the transfer of content BACK from the machine that has been built, i.e. when it's completed. We then want to capture the extra bits downloaded during the mini setup phase of Windows (such as Office installs etc.) and put back on the key so it can be used for later deployments. This involves moving the delta from an online BC cache and then back to an offline BC cache on the USB stick. As the regular built in API's (PowerShell/WMI) does a 'full dump', that is not really a good option as we want the delta to be as fast as possible. So we are writing some code to do this for us. The same code can then be used to input data from the USB key to a network device to populate it's cache, on a delta level, block by block. For network connected devices we are planning a feature in our upcoming project - codename "Chain home system" which will allow for syncing client caches between each other, so machine A can auto feed machine B etc. More on that in a later post..

This is cool, what can it help with?

Areas where USB media can help:

1. Building on separate "build networks" with few or no peers available.
2. Building when different machines require different setups, making caching harder.
3. Allow technicians to use a personal USB key to build machines in the field.
4. Allows for a simple way to seed locations via computers before being shipped.
5. Allows for a good way to populate build media to be used at home without worrying about lost USB key

OK  - What Next?

Get the OSD Toolkit and give it a try! - Download the latest Here: https://2pintsoftware.com/products/osd-toolkit/