The Magic of Vagrant: Automating My Home Lab with Infrastructure as Code
Virtual machines are a great way to isolate applications, but managing the lifecycle of a VM is often done manually. In this post, I am going to show you how to manage an entire VM using Vagrant. By the end of this article, I will show you how to launch a virtual machine, run commands inside it, and manage its entire lifecycle strictly through the command line.
Using your laptop or PC for your software tasks is convenient, and oftentimes, that is enough to run almost anything. However, in the software development lifecycle, running an app in an isolated environment is the preferred approach because it provides benefits such as enhanced security and granular permissions. With containers, this works great since applications run in their own lightweight environments. However, virtual machines are also ideal for a variety of tasks, like running Homebridge, Pi-hole, or a combination of applications. Just as you can manage containers through commands, you can also manage virtual machines through Vagrant commands like up, halt, and destroy.
My home computer is a PC, and I run Windows Pro for my day-to-day activities. While working on my self-hosted apps, I wanted to add Homebridge. After setting it up, I immediately realized that I was running into a port conflict on UDP port 5353, the standard port for Multicast DNS (mDNS) discovery. It seems that both Windows and Google Chrome also use this port for discovering casting devices. I decided to let that app go, but I encountered a similar issue raised with Pi-hole over port 53, because certain Windows services (like Internet Connection Sharing or DNS Client via svchost.exe in process lists) can occupy port 53. I decided to stop fighting these local conflicts and moved to virtual machines. Since Windows already has a great native hypervisor in Hyper-V, I decided to use that. A huge plus is that you can also manage Hyper-V virtual machines via PowerShell, which helps because I like to keep things under version control.
Hyper-V is simple, has a clean UI, and is easy to use. You can create a virtual machine with a few clicks or a few lines of PowerShell code. I started writing my own PowerShell scripts to manage the VM lifecycle. In Hyper-V, you can create a machine from scratch or import a pre-configured "Hyper-V appliance." There is actually a Homebridge Hyper-V appliance maintained by the community, and once imported, the UI is accessible immediately. For Pi-hole, however, you need to create a new VM, followed by a one-step automated install script. My own PowerShell scripts could do this, but I realized my manual approach had three main flaws:
Complexity: Separating code from configuration is a best practice, but managing it manually was becoming a chore.
High Maintenance: I know myself; six months from now, I won't remember the logic behind these scripts or how to troubleshoot them.
Lack of Portability: These scripts are tied to my current environment and simply won't work if I shift my hardware or OS.
Vagrant is a command-line utility that provides a unified way to manage the virtual machine lifecycle. What's great about it is that you can version-control everything you do, and it is a tool that is widely accepted and future-proof. Because Vagrant abstracts the hypervisor, it works with Hyper-V, VirtualBox, and even cloud providers. It gives you a single API; behind the scenes, it integrates with the hypervisor SDKs. If I choose to move from Windows to Proxmox or Google Cloud, my virtual machines are ready to go with only a few lines of code changes.
Ready to dive in? Below, I am going to introduce Vagrant and how to install it on Windows. After that, I will show how to set up your first VM and even configure the VM with a purpose through provisioning. For the rest of the article, I use Windows and Hyper-V as the VM hypervisor. So, let's get started.
🧩 The Vagrant Ecosystem
Vagrant is a command-line utility tool to manage virtual machines. It helps create and configure reproducible virtual machines that you can use for any kind of work. It does the same job as Docker CLI, but instead of containers, it manages virtual machines. To understand Vagrant, you need to learn the following concepts.
Boxes: A box is the packaging format that Vagrant uses to share virtual machines. It is a pre-packaged base image that allows skipping OS installation screens. It is the template (like a Docker image), and a virtual machine is the copy of the box created from this image.
Vagrantfile: This is a file that describes a VM's configuration and provisioning. It is your intent of what the VM is, how many resources the virtual machine has, what it does, and how it gets provisioned. A Vagrantfile is the recipe for the VM. It is written in Ruby and looks easy.
Providers: Providers are infrastructure systems that allow you to create virtual machines. While VirtualBox, Hyper-V, and Docker are built-in providers, Vagrant supports all major cloud providers like Google Cloud, Amazon Web Services (AWS), Microsoft Azure, etc. Unlike VirtualBox, Hyper-V usually requires Administrator privileges for Vagrant to work. There are more than enough providers here for what you will ever need.
Provisioners: Provisioners are ways to install software or make changes to the VM. Vagrant offers different ways to do provisioning, from SSH into a VM to Ansible playbooks. For simplicity, I stick with the shell provisioner in this post. It allows you to run standard bash commands inside the VM via SSH, and it feels like magic.
Vagrant Public Registry: This is HashiCorp's Public Registry where pre-configured boxes are available for download. It shows detailed information about each box. It is very similar to Docker Hub.
The Vagrant workflow starts with the Vagrantfile. Here is how these concepts work with each other under the hood.
The Intent: This is the desired state defined in the Vagrantfile and the definition of the VM.
The Fetch: Vagrant checks the local machine first, and if not found, it pulls the Box in the Public Registry.
The Build: Vagrant tells the Provider (in our case Hyper-V) to create a new VM using that Box.
The Setup: Once the VM is running, the Provisioner (Your script) goes inside and installs your software.
Using Vagrant is very easy; there are only a handful of commands you need to know to get up and running.
vagrant up: This is the command to create the VM and start configuration through provisioning. It checks whether you have the box locally; if not, it downloads it from the registry, creates the VM, and starts provisioning. Similar to docker compose up, it gets the VM into the desired state.
vagrant status: Prints the state of the machine in your console and exits.
vagrant halt: This command stops the machine. VM is gracefully shut down.
vagrant ssh: This command is your way to get inside the VM and open a shell, similar to docker exec -it <container_name> /bin/bash in Docker.
vagrant destroy: This command stops the VM and deletes all the resources created for this VM. It destroys resources created for the VM, but doesn't remove the box used in the vagrant up command.
Under the Hood: The Build Logic
Vagrant is a sophisticated command-line tool that does a lot internally to create VMs. The basic idea is that Vagrant manages boxes so they become VMs through the instructions in a Vagrantfile. The following steps explain how Vagrant builds a VM.
The Box Metadata and Discovery
The Decompression and Import Logic
The Layering Strategy
Hardware Virtualization Mapping
The "Insecure" SSH Handshake
To explain this better, I will share path examples. Basically, Vagrant uses two main folders during usage.
Vagrant home folder (Home folder): This is Vagrant's default app directory, and it is stored in the home directory as
~/.vagrant.d/.VM folder (Project folder): This is where your Vagrantfile is. I assume this is stored in
/path/to/MyProject/Vagrantfile. If MyProject is the project folder, Vagrant will use the/path/to/MyProject/.vagrant/directory.
1. Box Metadata and Discovery
When Vagrant runs and gets the box name from the Vagrantfile, it checks the box in the home folder, ~/.vagrant.d/boxes, and if the box doesn't exist, it downloads it from the Vagrant Public Registry. A Vagrant box is a tarball (compressed archive) containing 3 parts.
Virtual Disk: A provider compatible disk format. In Hyper-V, this is the VHDX file that essentially is the disk space.
The metadata.json: A metadata file that stores information about the provider type and the architecture.
The Vagrantfile: The configuration of the Box that sets Box specific information. This is very similar to the Vagrantfile we make ourselves for our VMs.
2. Decompression and Import Logic
After the box is downloaded, Vagrant doesn't use it directly for VM creation. Because Hyper-V doesn't offer direct interaction, Vagrant performs operations through PowerShell and Windows Management Instrumentation (WMI) to execute actions in Hyper-V. For example, Vagrant uses commands like Import-VM and Add-VMHardDiskDrive to create the VM in Hyper-V.
The provider assigns a Universal Unique Identifier (UUID) to the newly created VM instance. Vagrant gets this value and stores it in your project directory. The filepath looks like .vagrant/machines/<vmname>/hyperv/id. This ID is the key Vagrant uses to know the VM in the provider is linked with your VM.
3. The Layering Strategy
Vagrant uses differencing disks when cloning the disk from the home folder to the project folder. Vagrant creates a differencing disk in the project folder for the VM. You can control this behavior from VM configuration via vm.provider.linked_clone. The advantage is significant disk space savings because the entire VHDX file is not copied to the project folder; instead, a new small layer is created in the project folder, and the home folder VHDX file is linked to this small layer as its parent. In my local VMs in Hyper-V, this is clearly visible in the UI (VM Settings/Hard Drive/Inspect).
The image below shows the virtual disk file properties of two VMs. On the left, base-server-01 VM with linked clone creates a Differencing virtual hard disk on top of the home folder VHDX file, and on the right hub-01 VM creates a clone of the home folder VHDX file.
4. Hardware Virtualization Mapping
Vagrant maps the Vagrantfile configuration to the materialized VM in the provider, such as physical hardware, Mac Address Generation, etc.
Mac Address Generation: If the Vagrantfile doesn't configure a MAC address, Vagrant generates a unique MAC address for the VM to prevent network collisions. For Hyper-V machines, these MAC addresses start with
00155Das this is the identifier for Hyper-V virtual machines.Memory/CPU Sizing: Through Vagrantfile provider settings, Vagrant configures VM memory and CPU values in Hyper-V.
Networking: If Vagrantfile configures the network switches, Vagrant will use that network. If not and multiple network switches exist, Vagrant will prompt in the terminal to choose one.
5. The "Insecure" SSH Handshake
The VM is already running and, because it is headless (no UI), Vagrant performs an internal handshake to connect to the machine. This handshake happens over SSH. By default, Vagrant needs initial credentials for the first SSH login. Many boxes include Vagrant’s publicly known insecure public key in the guest’s authorized_keys file, and Vagrant uses the matching insecure private key from the host for first access.
Therefore, the first step is that Vagrant uses the insecure key pair to connect to the VM
Vagrant then generates a new SSH key pair and replaces the insecure key entry in the VM's
authorized_keysfile. This is similar to the SSH setup we use for GitHub accounts or for logging into remote computers via SSH.Once the private key is set, it becomes the tunnel for Vagrant to perform any kind of operation on the VM side. This is amazing. The private key is stored in the project folder, in the
/.vagrant/machines/<vmname>/hyperv/private_keyfile.
The Magic of Automated Provisioning
Just because the provisioning felt like magic to me, I wanted to dig in and find out how it works. Here we are going to examine the magic of provisioning the VM.
For provisioning to work, the VM needs to be up and running, and a successful SSH connection must be established. At this point, the VM is ready for provisioning. Provisioning works in five phases.
Validation
Connection
Upload
Execution
Cleanup
1. Validation
Vagrant has a range of provisioners, so it uses a plugin architecture where every provisioner implements a standard interface. For example, the interface has a provision method, which receives the machine object and the configuration settings.
This allows you to define the provisioner simply by adding the provisioner name in the Vagrantfile, like vm.provision "shell", vm.provision "ansible", etc.
2. Connection
Before running any code, Vagrant performs checks on the VM. These are called Guest Capability Detection. These checks are:
sudo availability
Identifying the guest OS: Vagrant sends a small probe via SSH and detects the correct way to perform tasks like uploading files and executing commands, specifically for that OS.
3. Upload
Provisioners use the Communicator (usually this is SSH) to interact with the VM.
File uploads: Provisioning files are uploaded to a temporary directory inside the VM, via SCP or SFTP.
Environment Injection: Environment variables defined in the Vagrantfile are exported into the shell session before the main script starts.
4. Execution and Idempotency
This is where the actual scripts run inside the VM. Vagrant triggers the execution of the uploaded script.
Shell provisioner: Runs the script and captures STDOUT and STDERR in real-time and streams it to the Vagrant output in the host so that you can see the progress
Other provisioners (Ansible/Chef): These tools can run in Guest or Host mode; therefore, Vagrant performs the following actions for each:
Guest mode: Installs the tool inside the VM, and then runs the script via the tool within the VM
Host mode: The tool must be available in the host so that Vagrant uses it. Vagrant uses the tool on the host machine via SSH.
Vagrant runs the provisioner scripts once, only on the first vagrant up command. The second time the up command runs, provisioner scripts won't run by default. This is a safety-gate behavior to make the up command achieve idempotency. You can change this behavior in the configuration of the provisioner script via the "run" argument. You can run a provisioner script with the run: "always" or run: "never" options. To run provisioner scripts intentionally, you don't need to destroy the machine and start from scratch. You can simply run vagrant provision or vagrant up --provision to trigger the provisioning scripts again.
5. Cleanup
After execution completes, Vagrant performs cleanup, removes temporary files, and closes the SSH connection to terminate the provisioning session. The VM continues to be in the running state after provisioning finishes.
🛠️ Getting Started on Windows
Requirements
Before getting started, ensure you have the following:
📋 Prerequisites
OS: Windows Pro, Enterprise, or Education (Hyper-V is not available on Windows Home).
Hardware: Virtualization (VT-x/AMD-V) must be enabled in your BIOS/UEFI.
Access: A user account with Administrator privileges.
🛠️ Preparation
Create a project folder (e.g.,
D:\MyProject\my-ubuntu-box).Open PowerShell or Terminal as an Administrator.
Install Vagrant to Your Computer
Install Vagrant on your host computer where Hyper-V is enabled. Through administrator privileges, Vagrant will run PowerShell commands to execute actions in Hyper-V.
Vagrant is an executable binary and can be downloaded from HashiCorp's Install Page, though I prefer installing it through WinGet (Windows Package Manager), Windows' native command-line tool and client interface for installing apps.
winget install -e --id Hashicorp.Vagrant
After this command, you might need to restart your terminal and even your computer to use Vagrant from the command line.
Does Vagrant Need to Be Installed Inside the VM?
Vagrant is the tool that manages the lifecycle of VMs, so it doesn't need to be installed inside the VM. The VM is the product built by Vagrant, and Vagrant is the tool that orchestrates the process.
🚀 Launching Your First VM
The first thing to do is select a box to use in your hypervisor. As mentioned, these boxes are pre-configured and specific to providers. However, there are boxes configured for many providers, such as the generic boxes in Vagrant Public Registry. Because I wanted an Ubuntu machine, I went with generic/ubuntu2204, which had the LTS at the time of this article.
When I was writing my own PowerShell scripts, I initially made a mess by mixing the configuration with the Hyper-V automation scripts. With Vagrant, configuration is a simple Ruby file. In its simplest form, it looks like this.
Vagrant.configure("2") do |config|
config.vm.box = "generic/ubuntu2204"
end
At this point, you are ready to launch your machine. The next step is to start the VM.
Launching
Open up a terminal where your Vagrantfile is, and simply do vagrant up.
PS D:\vm-home\vms\my-ubuntu-box> vagrant up
Bringing machine 'default' up with 'hyperv' provider...
==> default: Verifying Hyper-V is enabled...
==> default: Verifying Hyper-V is accessible...
==> default: Importing a Hyper-V instance
default: Creating and registering the VM...
default: Successfully imported VM
default: Please choose a switch to attach to your Hyper-V instance.
default: If none of these are appropriate, please open the Hyper-V manager
default: to create a new virtual switch.
default:
default: 1) External Virtual Switch
default: 2) Default Switch
default: 3) WSL (Hyper-V firewall)
default:
default: What switch would you like to use? 1
default: Configuring the VM...
default: Setting VM Enhanced session transport type to disabled/default (VMBus)
==> default: Starting the machine...
==> default: Waiting for the machine to report its IP address...
default: Timeout: 120 seconds
default: IP: 192.168.1.139
==> default: Waiting for machine to boot. This may take a few minutes...
default: SSH address: 192.168.1.139:22
default: SSH username: vagrant
default: SSH auth method: private key
default:
default: Vagrant insecure key detected. Vagrant will automatically replace
default: this with a newly generated keypair for better security.
default:
default: Inserting generated public key within guest...
default: Removing insecure key from the guest if it's present...
default: Key inserted! Disconnecting and reconnecting using new SSH key...
==> default: Machine booted and ready!
Connecting
After Vagrant provisions the VM, you can check the status of the machine with vagrant status or simply check in the Hyper-V UI if the machine is up and running. Once it is ready, to connect to the machine, do vagrant ssh.
PS D:\vm-home\vms\my-ubuntu-box> vagrant status
Current machine states:
default running (hyperv)
PS D:\vm-home\vms\my-ubuntu-box> vagrant ssh
vagrant@my-ubuntu-box:~\( echo hello, \)(hostname)!
hello, my-ubuntu-box!
vagrant@my-ubuntu-box:~$
At this point, you are inside the VM and ready to do anything. You can now install any apps and alter the machine; it's up to you. A Pi-hole install is one command away. That is, if you want to configure the machine manually. Or, you can provision it the same way, with SSH, but in an automated way through provisioning.
I am going to destroy the machine by calling vagrant destroy to start from scratch for provisioning.
PS D:\vm-home\vms\my-ubuntu-box> vagrant destroy
default: Are you sure you want to destroy the 'default' VM? [y/N] y
==> default: Stopping the machine...
==> default: Deleting the machine...
Provisioning
Vagrant provides provisioning configuration in the Vagrantfile through vm.provision space. The script below prints the host name of the VM after it is created via vagrant up.
Vagrant.configure("2") do |config|
config.vm.box = "generic/ubuntu2204"
# Set the machine name
config.vm.hostname = "my-ubuntu-box"
# Simple provision script
config.vm.provision "shell", inline: "echo hello, $(hostname)"
end
After running vagrant up, Vagrant will output the following.
PS D:\vm-home\vms\my-ubuntu-box> vagrant up
Bringing machine 'default' up with 'hyperv' provider...
==> default: Verifying Hyper-V is enabled...
==> default: Verifying Hyper-V is accessible...
==> default: Importing a Hyper-V instance
default: Creating and registering the VM...
default: Successfully imported VM
default: Please choose a switch to attach to your Hyper-V instance.
default: If none of these are appropriate, please open the Hyper-V manager
default: to create a new virtual switch.
default:
default: 1) External Virtual Switch
default: 2) Default Switch
default: 3) WSL (Hyper-V firewall)
default:
default: What switch would you like to use? 1
default: Configuring the VM...
default: Setting VM Enhanced session transport type to disabled/default (VMBus)
==> default: Starting the machine...
==> default: Waiting for the machine to report its IP address...
default: Timeout: 120 seconds
default: IP: fe80::215:5dff:fe01:7824
==> default: Waiting for machine to boot. This may take a few minutes...
default: SSH address: 192.168.1.141:22
default: SSH username: vagrant
default: SSH auth method: private key
default:
default: Vagrant insecure key detected. Vagrant will automatically replace
default: this with a newly generated keypair for better security.
default:
default: Inserting generated public key within guest...
default: Removing insecure key from the guest if it's present...
default: Key inserted! Disconnecting and reconnecting using new SSH key...
==> default: Machine booted and ready!
==> default: Setting hostname...
==> default: Running provisioner: shell...
default: Running: inline script
default: hello, my-ubuntu-box
PS D:\vm-home\vms\my-ubuntu-box>
At this point, you have fully configured a VM successfully and purposed it for a specific task.
🔍 A Note about the IPv6 Address
Did you notice the initial IPv6 address on the second boot? That is a link-local IPv6 address created by Ubuntu. When the machine boots, the VM reports its address to Hyper-V faster than the IPv4 address received from the router's DHCP server. Therefore, this happens because of a race condition between the two, and the first reporter wins. But Vagrant is smart enough to use the IPv4 address, so this is not an issue.
🏗️ Final Thoughts: The Path to Infrastructure as Code
In this post, we built and managed a Hyper-V virtual machine end-to-end using Vagrant and the command line. We covered the key parts of the workflow: boxes, providers, provisioning, SSH handshake behavior, and how Vagrant turns a simple Vagrantfile into a running VM you can reproduce. Some parts of this post may not be accurate for other providers, but the core ideas should still apply.
For me, the biggest value is portability and repeatability. Today I use Windows and Hyper-V, but the same approach can be adapted to other providers with minimal changes. That is the real power of Infrastructure as Code: your environment becomes documented, version-controlled, and easy to rebuild when you need it. In a home lab environment, we usually set things up and move on, then forget what we did later. With Vagrant, you can always go back to your Git history and see what changed. If you need to update something, you can change the Vagrantfile and run vagrant up again to apply it.
If this helped, share your setup or questions in the comments. The code from this post and additional examples are in my GitHub repository. Give it a star if you find it useful.
In the next post, I’ll move from a base server to an application host and show how to spin up a dedicated Docker VM that stays isolated while remaining reachable on the home network.
Thanks for reading.
Metin Senturk