We ignored the dangers of containers 30 years ago, and we're doing it - Lacework

We ignored the dangers of containers 30 years ago, and we’re doing it

Allie Fick, Security Reporter

October 24, 2022

Abstract architectural photo shot from the ground. Features a lot of modern windows and steel.There’s no arguing that containers changed the world—but did they change it for better or worse? Shipping containers and software containers, although introduced decades apart, enabled companies to deliver new things to customers faster than ever before. 

While businesses were distracted by shipping containers and the instant gratification and revenue that came with them, criminals and adversaries were exploiting the dangers they introduced. The history of shipping containers can help us avoid making the same mistakes with software containers, but only if the world is willing to pay attention to the lessons we learned.   

What is a container? 

In general, a container is a portable receptacle that holds things inside it. They make it easier to transport things from one location to another in a simple enclosure. 

With software containers, the receptacle is a software unit (a small piece of code). The things it holds together are all of the data and code needed to run an application. 

With shipping containers, the receptacle is a standardized box, usually made of steel. The things inside are goods of various forms and sizes.  

Getting what you want wasn’t always this simple 

Today, when we want something, we order it, and it arrives a few days later. But do you ever think about where those items come from? 

When you’re shipping something from one location to another, the item often has to go through a variety of transportation methods, from vehicles to cargo ships to airplanes. This was a long and expensive process that required a lot of manual labor until shipping containers were introduced in 1956. The containers reduced the need for manual labor and made shipping cheaper, faster, and more efficient. They allowed companies that previously only shipped their products domestically to ship them internationally while keeping the contents of the containers intact. For consumers, shipping containers made new products available at accessible prices. 

Software containers made applications like you know them today

This is a lot like containerization of software.  

Do you remember a time before there was a store where you could download an application for anything you could imagine? We take it for granted now, but it wouldn’t be possible without containers. 

These containers make it easier to package and transport software apps because developers don’t have to manually move code or worry about whether the app will work in different environments. 

The concept of containers isn’t new—however, creating and using containers historically required a lot of manual command writing and specific expertise. This all changed when Docker was introduced in 2013, making containers much easier to create and distribute. 

Docker created a standardized image, which made containers more user-friendly and appealing to developers. An image is a read-only file that can be used as a template to create containers. 

Kubernetes, a container orchestration software, took this a step further by making it possible to create and manage apps that use hundreds of containers.

Security: a selling point gone wrong

“The history of the shipping container is humbling,” economist Marc Levinson wrote in his book The Box. 

Improved security was originally one of the shipping container’s selling points; however, it wasn’t long before the container became an international security threat.

Packing items inside of a locked container was intended to prevent goods from being stolen or damaged, but in the 1980s, thieves began to discover that they could easily take advantage of the reliability and anonymity of container shipping. 

It wasn’t just the containers with high-value goods like jewelry or electronics that were at risk—if you have enough of anything, their value can add up. Even if it’s dolls based on a movie character, like the $92,000 worth of Freddie Krueger dolls packed in a cargo shipping container stolen in Carson, California in 1989. Thieves broke through a warehouse fence and stole an entire container, including the 3,700 dolls inside. It was packaged, ready to go, and easy to steal. 

This represents a wider issue that was happening at the time—containers were being stolen at ports, their surrounding industrial areas, and warehouses. Because the container shipping industry and ports were rapidly growing and changing, thieves were presented with more opportunities, sound familiar?. 

Container theft was popular because it was a low-risk, high-profit crime. Many warehouses only had fences and locked gates to keep people out, and companies found it too expensive to hire full-time security guards or install security systems. It was easy for thieves to go undetected in a chaotic environment with minimal security measures.  

Does this sound familiar to you? 

Not much has changed about the mindset of criminals today, with cybercriminals looking for low-risk, high-reward crimes

Nearly 35 years ago, the warehouse owner in California thought security measures were too expensive, but then found himself purchasing a $28,000 security system after he discovered the importance of security the hard way. Today, some business owners avoid making investments in security until after an attack has happened, even though the current average cost of a data breach is $4.35 million

And theft was just the tip of the iceberg. Many more dangerous security threats would come with shipping containers over the next four decades. 

A blessing for the economy, but a curse for security

Criminals found another way to take advantage of containers in the early 1990s when they began using containers to smuggle drugs and people internationally, which further increased security concerns. 

“This high-efficiency transportation machine is a blessing for exporters and importers, but it has become a curse for customs inspectors and security officials,” Levinson wrote. 

Not long after the use of shipping containers for smuggling gained traction, experts realized that terrorists could take advantage of containers in devastating ways. This became a major concern after the September 11, 2001 terrorist attacks on the US, when experts began to worry about adversaries placing a nuclear weapon in a container to attack the country. 

This parallels the benefits and downsides of software containers—they have led to cost savings, speed, and are an integral part of the software development process. Unfortunately however, they are an appealing target for hackers. 

For example, images are used as blueprints, or templates, to create containers. Images are published online on registries, like Docker Hub, where users can download them and fill in those templates with their own data to create applications. But if those images contain malicious or vulnerable code, the thousands of users who download them and use them to create their own containers or applications are at risk. On top of that, any users who download the application also have the vulnerable code on their machines. And because containers are a newer technology, not everyone, including many security teams, fully understand the threat

A layered approach to security 

To determine how to secure shipping containers, the government decided to look at the shipping industry as a whole. Not only did the containers themselves need to be scrutinized, but so did their origins, destinations, storage locations, and transportation methods. 

Since the 9/11 attacks, several federal initiatives have been implemented to encourage a layered approach to shipping container security, including container detection and monitoring capabilities, sensor technology, automated tools to improve data analysis, tools to detect anomalous activity patterns, and research to improve threat detection capabilities. 

Interestingly enough—and probably not coincidentally—these are the same tools and technologies we use to protect our data today. 

Did container screening in 2002 foreshadow “shifting left” today? 

The Container Security Initiative (CSI) was launched in 2002 to ensure that all containers headed to the US on ships were screened at foreign ports before they were loaded onto the vessels. The goal of the initiative was to both increase shipping container security while also avoiding trade interruptions by requiring containers to be scanned as early in the supply chain as possible. Today, US Customs and Borders Protection use automated tools and strategic intelligence to identify which containers are most at risk for terrorism as well as utilizing x-ray, gamma ray, and radiation detection technology to further screen those containers that are at risk. 

This is a lesson we apply to the software development lifecycle today, often referred to as “shifting left,” which means that we perform testing and incorporate security earlier in the software development life cycle instead of waiting until runtime. If we catch code misconfigurations as early as possible, for example, when the container images are created, we’re much better off than waiting until an application has been downloaded by hundreds of thousands of users worldwide. 

Catching vulnerabilities earlier saves both time and money—this applies regardless of which type of container you’re talking about. Tracking down each user who downloaded a vulnerable application would be nearly impossible. Just like finding every single person who purchased a broken product manufactured and shipped from another country would be a daunting task. 

Maximizing protection comes down to each individual port

While the federal government implemented security requirements nationwide, individual ports were taking it upon themselves to strengthen their security by using practices customized to their unique location and needs. Which is why, in 2008, the National Institute of Justice (NIJ) funded a forum to research some of the best and most promising security practices implemented by local ports in the US. 

Software container platforms, like Docker, have some security features baked in. But containers are used by so many different companies with different use cases, organizations need to enhance that security, and that’s their own responsibility. Similar to how the individual ports customized their security routines, every company needs to implement security solutions based on their needs. 

The NIJ-funded forum discovered that local ports’ shipping container security practices fit into five general categories—awareness, prevention, preparedness, response, and recovery—and the best strategies incorporated all five. While their study was published 14 years ago, those categories are more relevant than ever—each is referenced several times in the recently released Cybersecurity and Infrastructure Security Agency’s (CISA) 2023-2025 Strategic Plan, which explains how the US will reduce risk and build resilience to cyber and physical threats. 

Awareness is just the beginning

The research group found that the local ports with the most successful awareness strategies encouraged collaboration between port stakeholders. They developed protocols to make sure different groups in the port community were talking about security concerns. In the IT industry, this is like the idea behind the DevSecOps philosophy. Development, operations, and security teams have different perspectives, motivations, and skills, and in order to succeed, they need to work together. 

But you can’t spread awareness without visibility. Before you can communicate your security concerns, you need to see what’s happening with your containers. What is your company using containers for? Who uses them? What’s inside them? Being aware of container-related events, communication, new connections, and images across the application/process, container runtime, and orchestration layers will make it easier to identify threats. 

Access control prevents malicious users from entering 

Prevention was the most commonly implemented strategy at the ports. While many used traditional access control methods like IDs and badges, biometric and other innovative technologies were increasing in prevalence along with simple measures like identification numbers stenciled on workers’ uniforms. Some ports also required employees to travel from one building to another on a shuttle to prevent them from accessing other areas unauthorized. 

In the cloud, identity and access control is essential to make sure that only those who are authorized can access the information in your containers. And just like ports stepped up security by requiring licenses/badges combined with biometric technologies, identity and access management (IAM) is necessary to make sure the right people are accessing the right information. 

This became even more clear when Docker was hit with a container attack in 2020. This breach was carried out by a group who searched specifically for insecure containers that were operating with lax authentication policies. With access, they installed a crypto mining program that spread to as many as 6,000 images. 

Detection finds anomalous behavior quickly

Many ports installed detection equipment to supplement the security screenings being conducted by the government worldwide. And the most effective equipment used video analytics and algorithms to detect anomalous behavior. Because ports had continuous motions and people traveling in and out, simple sensor technology to detect motions would generate too many alerts. However, technology to detect unusual behavior like a ship or vehicle stopping in a different place could help identify risks. 

This is exactly why behavioral threat detection is so important to protecting software containers. Containers are constantly changing and moving, so the best way to understand when something is a real risk is to leverage analytics and machine learning to understand normal behaviors in your environment, so you can detect and respond to anomalous ones. 

What we know now

The history of shipping containers taught us that innovative technologies intended to solve problems can create new, even more dangerous ones. But the good news is that a layered security approach and a strong understanding of your environment can prevent exploits from occurring. 

While basic security measures employed by the software you use to create and manage containers are great to have, organizations need to take security a step further to secure their environments. In the same way individual ports found tools, processes, and systems that improved their shipping container security, each company is ultimately responsible for protecting their data. Getting a comprehensive view into all of your container activity and building a baseline that’s normal with your containers will help you quickly identify malicious behaviors and vulnerabilities. 

For more information on container security and how to protect your cloud environment, check out our guide on container security essentials.