What!? Moving Big Data With A Truck?

You might have heard the term "Big Data" being discussed. The large amount of data that companies are processing daily. But have you ever thought how companies take care of such large amounts of data? How the data is migrated and moved between servers?

Imagine that large amount of data we're generating every day is increasing exponentially every year and the amount of data that was considered extremely large two decades ago is now being generated in a matter of seconds. The data we've had in our largest storages - can today be kept in the pocket.

But now we've got new challenges and the data is just increasing and today, you'll see something interesting that Amazon Web Services are providing for their customers.

The Snowmobile, The Big Data Truck

Well, as companies are expanding and migrating from on-premise to cloud providers - they also have to bring their data with them.

Larger companies that have huge amount of data to transfer and if they would transfer their data over the network cable, it would have taken ages (literally).

Companies solved this with larger disks that can carry up to terabytes of data, which is much - but not enought for some of them. This is where Amazons Snowmobile data 14 meters truck comes into the picture.

They have created this solution to optimize cloud migration (moving data to the cloud), and it works. In their truck, they can move up to 100 PB data in one way, and if you would do some rounds to move exabytes of data to their cloud, they increased the speed by years. Instead of moving exabytes of data for 26 years, they can now do it in 6 months instead.

Their solution works like this:

Order the Snowmobile

You make an order of a Snowmobile and they arrive with the truck.

Load the truck

You connect the truck with the fiber cable to your datacenter and let the process work.

Fill your AWS data

They take your data in the truck and some weeks later you'll see your data in the cloud. Simple as that.

So How Much Is It Really? Let's look at a Petabyte-level

Well, as we know, the amount of data expanding expremely fast and some of the largest companies are generating hundreds of petabytes of data daily. It's hard to grasp how much that is, but if we would image it in MP3 songs, it would round up to 200 million songs.

Just imagine how much data is stored in Facebook servers on their 2.7 billion users and how much every image, chat conversation and post has to take.

Facebook is not the only one which has billions of people using the software and the data is flowing different directions all day, every day, every week, every year. Banks, stock markets, other social media channels, Amazon data storage, Google searches – all of them.

This is huge and it’s just increasing. With the high volume of software services, consumers have everything online and the datacenters are taking care of everything in their machinery. Almost every website and business are regularly collecting large amounts of data, all of which can be classified as Big Data, and that’s big.

The data is important for the companies

In 2020, Big Data is essential for companies. These large amounts of collected data provide excellent insight into the user experience.

Companies such as Netflix, collect all the data points from their users to recommend you the movies that they liked, which you might like as well. Or large companies such as Amazon, where the product recommendations just know what you want – before you even know that you want it.

This is processed by huge amount of data and it’s tweaking the algorithm and suggestions for you, so you make a purchase. The customers are getting a greater experience and the companies increase their revenue. With Big Data there’s tons of value and many companies sees it as the new oil. And the AWS Snowmobile is now the 2020 type of oil truck.

Besides recommendations, predictions and better user experience on websites, Big Data is also useful for advertising, where companies can process users' data and show them targeted advertisements. Other industries that will have an advantage of Big Data is IoT (Internet of Things).

With millions of smart devices in industries, cities, homes and roads – data is and will be collected. Everything to keep the devices smart and increase the predictions for different services, solutions and products.