What is Network Automation?

Network Automation and NetDevOps are hot topics in the network engineering world right now, but as with many new concepts, it can be confusing to decipher the meaning from the noise in the quest to achieving optimal efficiency and agility of network operations.

A useful starting point would be to first define what network automation is not:

• Network automation is not just automated configuration generation or inventory gathering
• It is not just using the same network management system (NMS) as today but faster
• It is not just performing patching and OS upgrades faster, or network engineers suddenly becoming software developers
• Network automation is not going to work in isolation of changing lifecycle and deployment processes, nor is it a magic toolbox of all-encompassing applications, frameworks and code.

At CACI, we view network automation as both a technology and a business transformation. It is as much a cultural shift from legacy deployment and operations processes as it is a set of tools and technology to implement speed, agility and consistency in your network operations. Infrastructure is changing fast, and with Gartner reporting 80% of enterprises will close their traditional data centres by 2025, the only constant in networking is that change will persist at faster clip.

So, how does Network Automation work? What differentiates network automation from NetDevOps? What difference can it make to modern IT operations, and which best practices, technologies and tools should you be aware of to successfully begin your network automation journey?

How does Network Automation work? 

Network Automation implements learnings from DevOps developments within the software development world into low-level network infrastructure, using software tools to automate network provisioning and operations. This includes techniques such as:

• Anomaly detection
• Pre/post-change validation
• Topology mapping
• Fault remediation
• Compliance checks
• Templated configuration
• Firmware upgrades
• Software qualification
• Inventory reporting.

In understanding how these differ from traditional network engineering approaches, it is important to consider the drivers for network automation in the post-cloud era – specifically virtualisation, containerisation, public cloud and DevOps. These technologies and approaches are more highly scaled and ephemeral than traditional IT Infrastructure, and are not compatible with legacy network engineering practices like:

• Using traditional methodology to manage infrastructure as “pets” rather than “cattle”
  • Box-by-box manual login, typing CLI commands, copy-pasting into an SSH session, etc.
• “Snowflake networks” which don’t follow consistent design patterns
• Outdated (or non-existent) static network documentation
• Lack of network validation and testing.

Network automation aims to change all this, but to do so, must overcome some obstacles:

• Cross-domain skills are required in both networking and coding
• Some network vendors do not supply good API or streaming telemetry support
• Screen scraping CLIs can be unreliable as CLI output differs even between products of the same device family.
• Cultural resistance to changes in both tooling and practice
• Lack of buy-in or sponsorship from the executive level can compound these behaviours.

What differentiates network automation from NetDevOps? 

You may also have heard of “NetDevOps” and be wondering how – or if – this differs from network automation. Within CACI, we see the following key differences:

We often see our clients use a blend of both in practice as they go through the automation adoption curve into the automation maturity path, from ad-hoc automation, through structured automation, into orchestration and beyond:

What difference can network automation make to modern IT operations? 

Network automation aims to deliver a myriad of business efficiencies to IT operations. This has proven to be transformational across our wide and varied client base, with improvements demonstrated in the following ways:

Increased efficiency 

Much of networking is repetition in differing flavours – reusing the same routing protocol, switching architecture, edge topology or campus deployment. A network engineer is often repeating a task they’ve done several times before, with only slight functional variations. Network automation saves time and costs by making processes more flexible and agile, and force-multiplying the efforts of a network engineering task into multiple concurrent outputs.

Reduced errors 

Networking can be monotonous, and monotony combined with legacy deployment methodology can cause repetition of the same error. Network automation reduces these errors – particularly in repetitive tasks – to lower the chances of reoccurrence. When combined with baked-in, systems-led consistency checking, many common – but easily-avoidable – errors can be mitigated.

Greater standardisation

Networks are perhaps uniquely both the most and least standardised element of the IT stack. While it is easy to have a clean “whiteboard architecture” for higher-level concerns such as application development, the network must often deal with the physical constraints of the real world, which, if you’ve ever tried to travel to a destination you’ve not been to before, can be messy, confusing and non-sensical. Network automation ensures the starting point for a network deployment is consistent and encourages system-level thinking across an IT network estate over project deployment-led unique “snowflake” topologies.

Improved security 

Increased security often comes as a by-product of the standardisation and increased efficiency that network automation brings. Most security exploits are exploits of inconsistency, lack of adherence to best practice or related – which ultimately pivot around “holes” left in a network (often accidentally) due to rushing or not seeing a potential backdoor, open port, misconfiguration or enablement of an insecure protocol. When combined with modern observability approaches like streaming telemetry and AIOps, network automation can help enforce high levels of security practice and hardening across an IT estate.

Cost savings

Given its position as the base of the tech stack, the network is often a costly proposition – with vertically-integrated network vendors, costly telco circuit connectivity, expensive physical world hosting and colocation costs, and so on – the network is often a “get it right first time” endeavour which can be cost-prohibitive to change once live and in service. Network automation encourages cost savings through the creation of right-the-first-time and flexible network topologies and in performing design validation which can minimise the amount of equipment, licensing, ports and feature sets required to run a desired network state.

Improved scalability

As both consumer and enterprise expectations of scale are set by the leading web scalers of the world, the enterprise increasingly expects the flexibility to scale both higher and lower levels of the IT stack to larger and more seamless sizes, topologies and use cases. Network automation aids in achieving this through the enforcement of consistency, modularisation, standardisation and repeatability for network operations.

Faster service delivery

IT service delivery is increasingly moving away from being ticket-led to self-service, with the lower-level infrastructure elements expected to be delivered much faster than the traditional six-to-eight-week lag times of old. As telco infrastructure moves through a similar self-service revolution, so too does the enterprise network require the ability for self-service, catalogue-driven turn-up and modularised deployment. Network automation enables this by optimising network performance to the required parameters of newer services and applications in the modern enterprise.

What are the best practices for network automation?

Network automation is as much a cultural transformation as it is a technology transformation. Much as DevOps disrupted traditional ITIL and waterfall approaches, NetDevOps similarly disrupts current network engineering practices. We find the following best practices to be beneficial when moving towards network automation:

Choose one thing initially to automate

• Pivot around either your biggest pain point or most repetitive task
• Don’t try to take on too much at once. Network automation is about lots of small, repeated, well-implemented gains which instil confidence in the wider business
• People love automation, they don’t want to be automated. The biggest barrier to adopting automation will be keeping colleagues and stakeholders on-side with your efforts by showing the reward of that they provide to them and to the wider business.

Choose tooling carefully

• Stay away from the “latest shiny” and pick open, well-used tools with large libraries of pre-canned vendor, protocol and topology integrations, and human-readable configuration and deployment languages
• Maintain your specific business context during tool selection
• Think ahead for talent acquisition and retention – writing custom Golang provisioning application might be handy today, but you could struggle to get others involved if the author decides to leave the business.

Optimise for code reusability

• Build and use version control systems such as Git, GitHub and Azure DevOps from day one and encourage or even mandate their use
• Advocate for the sharing of functions, modules, routines and snippets written within code, runbooks, IaC and state files within scrapbooks and sandpits. The flywheel of productivity increases exponentially within NetDevOps as increasingly more “we’ve done that before” coding and practices accelerate the development of newer, more complex routines, IaC runbooks and functions
• Code should be written with reuse and future considerations in mind. While it may be tempting to “save ten minutes” so as to not functionise, modularise or structure code, this will catch up with you in the future.

Use templating for configuration generation

• Templating programmatically generates the vendor-specific syntax for a network device based on a disaggregated, vendor-neutral input format (such as Jinja2, Mako or Markdown) which is later combined with data (such as specific VLANs, IP Addresses or FQDNs) to generate the vendor-specific syntax (such as Cisco IOS, Arista EOS or Juniper Junos) for the network device
• The act of creating the templates has an added by-product of forcing you to perform design validation. If your design document doesn’t have a section covering something you need template syntax for, it could well be due for an up-issue
• Templates become a common language for network intent that are readable by all network engineers regardless of their individual network vendor and technology background, aiding in time to onboard new staff and ensuring shared understanding of business context around the IT network.

Which tools, frameworks and languages enable network automation? 

There are a myriad of network automation tools, frameworks, languages and technologies available today. Deciphering these can be confusing, but a good starting point is categorising the distinct types of network automation tooling available:

Network Configuration and Change Management (NCCM)

• Enable patching, compliance and deployment (rollout)
• Often align to network management systems (NMS) or BSS/OSS (Telco space)

Network Orchestration

• Enable programmatic device access (CLI, API, SSH, SNMP)
• Often align to DevOps engineering usage

Policy-based Automation

• Abstract network device box-by-box logic into estate-wide, policy-driven control
• Often align to industry frameworks and controls (SOC2, HIPAA, CIS, PCI/DSS)

Intent-Based Networking Systems (IBNS)

• Translate business intent through to underlying network configuration and policy
• Are starting to become the “new NMS”

It would be exhaustive to list all possible tools, frameworks and languages available today, but these are some of our most seen within our client base today. Our current favourites can be seen in What are the most useful NetDevOps Tools in 2023?:

Tools

• Terraform – An open-source automation and orchestration tool capable of building cloud, network and IT infrastructure based on input Infrastructure as Code (IaC) code via HCL (HashiCorp Configuration Language) that defines all attributes of the device and configuration blueprint required. Terraform is highly flexible and has a vast array of pre-built modules and providers for most network engineering concerns via the Terraform Registry.
• Ansible – An open-source automation and orchestration tool typically used to configure within the device rather than provision the underlying Baremetal or cloud infrastructure the cloud, network or IT device sits atop, which is based on input IaC code via YAML that defines the attributes and device configuration required. Ansible is versatile and has a large cache of pre-built runbooks and integrations for network engineering concerns via Ansible Galaxy.
• NetBox – The ubiquitous, open-source IP Address Management (IPAM) and Data Centre Infrastructure Management (DCIM) tool, which acts as the Network Source of Truth (NSoT) to hold a more detailed view of network devices, topology and state than could be achieved via alternative approaches such as spreadsheet or CMDB. NetBox is highly customisable, with a rich plugin ecosystem and customisable data models via YANG to adapt around business-specific topology data models.
• Git – The de facto version control system, which is the underlying application that powers GitHub and GitLab and supplies a mechanism to store IaC, configuration and code artefacts in a distributed, consistent and version-controlled manner. Git is pivotal in enabling the controlled collaboration on network automation activities across a distributed workforce while maintaining the compliance and controls required within the enterprise environment.

Frameworks 

• Robot framework: A generic test automation framework allowing network automation code and IaC runbooks to run through acceptance testing and test-driven development (TDD) via a keyword-driven testing framework with a tabular format for test result representation. It is often used in conjunction with tools such as pyATS, Genie, Cisco NSO and Juniper NITA.
• PEP guidelines: Short for Python Enhancement Proposals (PEP), these are to Python what RFCs are to network engineering, and provide prescriptive advice on setting out, using, structuring and interacting with Python scripts. The most commonly known of these is the PEP8 – Style Guide for Python.
• Cisco NADM: The Cisco Network Automation Delivery Model (NADM) is a guide on how to build an organisation within a business around an automation practice, addressing both the human aspect as well as some of the tooling, daily practices, procedures, operations and capabilities that a network automation practice would need to take traction in an IT enterprise landscape.

Languages

• Python: The de facto network automation coding language, utilised as the underlying programming language in tools from NetBox, Nornir, Batfish, SuzieQ, Netmiko, Scrapli, Aerleon, NAPALM and more, popularised by its extensive network engineering-focused library within PyPi. Python is the Swiss army knife of NetDevOps, able to turn its hand to ad-hoc scripting tasks through to full-blown web application development using Flask or API gateway hosting using FastAPI.
• Golang: An upcoming programming language, which benefits over Python in terms of speed via a compiler-based approach, parallel-execution, built-in testing and concurrency capabilities when compared to Python. On the downside, it has a significantly steeper learning curve than Python for new entrants into the realm of development and has far fewer network engineering library components available to use.

What does the future of network automation look like? 

The demand for network automation and NetDevOps professionals is undoubtedly on the rise, a trend that we at CACI expect to continue as budgetary pressures from the macroeconomic climate accelerate and trends like artificial intelligence (AI) begin to challenge the status quo and push businesses to deliver seamless, scalable network fabrics with more expectation of self-service and less tolerance of outage, delay or error. We see more of our clients moving up through the automation maturity path towards frictionless and autonomous network estates and expect this to accelerate through the coming years with ancillary trends such as NaaS (Network as a Service), SDN (Software Defined Networking) and NetDevOps set to continue and embed the NetEng Team firmly into the forthcoming platform engineering teams of tomorrow.

How can CACI help you on your network automation journey?

With our proven track record, CACI Network Services is adept at a plethora of IT, networking and cloud technologies. Our trained cohort of high calibre network automation engineers and consultants are ready and willing to share their industry knowledge to benefit your unique network automation and NetDevOps requirements. We are a trusted advisor that ensures every team member is equipped with the necessary network engineering knowledge from vendors such as Cisco, Arista and Juniper, along with NetDevOps knowledge in aspects such as Python for application Development, NetBox for IPAM and NSoT, Git for version control, YAML for CI/CD pipeline deployment and more.

Our in-house experts have architected, designed, built and automated some of the UK’s largest enterprise, service provider and data centre networks, with our deep heritage in network engineering spanning over 20 years across a variety of ISP, enterprise, cloud and telco environments for industries ranging from government and utilities to finance and media.

Get in touch with us today to discuss more about your network automation and NetDevOps requirements to optimise your business IT network for today and beyond.