Proxmox VE commands

Proxmox VE command lines for the shell, taken from the synopsis lists and worked examples of the Administration Guide, version 9.2.13 (Proxmox VE 9). Rows are grouped by task: virtual machines with qm, containers with pct, then backup, storage, cluster, Ceph, high availability, users, node upkeep, firewall and the API shell. Words in capitals such as VMID are placeholders; numbers like 300 or 100 are the guide's example IDs. qm list and pct list show the guests of a node.

353 commands in 19 sections. Checked row by row against Proxmox VE Administration Guide, version 9.2.13 (Proxmox VE 9) on . Edited by Jay.

Virtual machine lifecycle 19

VM and container IDs are integers from 100 to 999999999.

CommandWhat it does
qm listList virtual machinesCovers the VMs of the node the command runs on.
qm status VMIDShow VM statusAdd --verbose for the detailed output format.
qm create VMIDCreate a virtual machineHardware such as disks and network cards is added through options; --archive restores from a backup instead.
qm create 300 -ide0 local-lvm:4 -net0 e1000 -cdrom local:iso/proxmox-mailgateway_2.1.isoCreate a VM from an ISOThe VM gets a 4 GB disk on local-lvm at ide0, an e1000 network card and the ISO as CD-ROM.
qm start VMIDStart a virtual machineFor a VM under HA the request is forwarded to the HA stack automatically.
qm shutdown VMIDShut down a VM cleanlySends an ACPI power-button event and leaves the guest OS to shut itself down.
qm shutdown 300 && qm wait 300Shut down and waitThe second command blocks until the VM has actually stopped.
qm shutdown 300 && qm wait 300 -timeout 40Shut down with a wait limitGives up waiting after 40 seconds instead of waiting forever.
qm wait VMIDWait for a VM to stopBlocks until the VM is down; without a timeout it waits forever.
qm stop VMIDStop a VM immediatelyEnds the QEMU process at once, like cutting the power, so guest data can be damaged.
qm stop 300 -overrule-shutdown 1Force-stop over a pending shutdownAborts shutdown tasks still running for the VM before stopping it.
qm reboot VMIDReboot a virtual machineA full shutdown and start, so pending configuration changes take effect.
qm reset VMIDReset a virtual machine
qm suspend VMIDSuspend a virtual machineWithout --todisk the VM is only paused, not saved to storage.
qm suspend ID --todiskHibernate a VM to diskSaves the VM state to storage, and the next start resumes from it.
qm resume VMIDResume a suspended VM
qm unlock VMIDRemove a VM lockClears a lock left by an interrupted migration, snapshot or backup, for example after a power failure.
qm destroy VMIDDelete a VM and its disksRemoves every volume the VM owns together with its permissions and firewall rules.
qm destroy 300 --purgeDelete a VM from all configurationsAdditionally removes the VM from replication jobs, backup jobs and HA resource configuration.

Virtual machine configuration 15

CommandWhat it does
qm config VMIDShow VM configurationPrints the configuration with pending changes already applied; --current shows the values in effect now.
qm pending VMIDShow pending VM changesLists current and pending values side by side.
qm set VMIDChange VM optionsSets one or more options on an existing VM; several can be given in one call.
qm set VMID -onboot 1Start a VM at host bootSame as the Start at boot option on the Options tab of the web interface.
qm set ID --tags 'myfirsttag;mysecondtag'Tag a virtual machineSeveral tags go into one value, separated by semicolons.
qm set VMID --allow-ksm 0Disable KSM for one VMStops memory page merging for this VM; the option is on by default.
qm set 100 --hookscript local:snippets/hookscript.plAttach a VM hook scriptThe script is a snippet file referenced by its volume ID.
qm set VMID -hostpci0 00:02.0Pass through a PCI deviceHands the host device at address 00:02.0 to the VM as hostpci0.
qm set ID -hostpci0 NAMEAssign a mapped PCI deviceNAME is a cluster resource mapping rather than a raw PCI address.
qm set VMID -delete vmgenidDelete a configuration propertyThe -delete option removes a setting from the VM configuration, here the generation ID.
qm showcmd VMIDShow the QEMU command lineDebug aid; --pretty puts each option on a line of its own.
qm enroll-efi-keys VMIDEnroll updated Secure Boot keysAdds the Microsoft UEFI 2023 certificates to the EFI disk and only works on a stopped VM.
qm terminal VMIDOpen a serial terminalNeeds a serial device in the VM configuration, such as serial0: socket.
qm monitor VMIDOpen the QEMU monitor
qm sendkey VMID KEYSend a key to a VMThe key is written in QEMU monitor encoding.

Cloud-Init and guest agent 14

CommandWhat it does
qm set 9000 --ide2 local-lvm:cloudinitAdd a Cloud-Init driveThis CD-ROM drive is how the Cloud-Init settings reach the guest.
qm set 9000 --boot order=scsi0Boot from one disk onlyRestricts the boot order to scsi0, so the BIOS skips testing for a bootable CD-ROM.
qm set 9000 --serial0 socket --vga serial0Use a serial console as displayMany Cloud-Init images expect a serial console; if an image misbehaves, return to the standard display.
qm set 123 --sshkey ~/.ssh/id_rsa.pubSet the Cloud-Init SSH keyInstalls the public key used for authentication in the cloned VM.
qm set 123 --ipconfig0 ip=10.0.10.123/24,gw=10.0.10.1Set the Cloud-Init IP addressGives the first network device a static address and gateway.
qm set 123 --cipassword PASSWORDSet the Cloud-Init password
qm set 9000 --cicustom "user=local:snippets/userconfig.yaml"Use a custom Cloud-Init fileReplaces the generated user config with a snippet that must exist on every node the VM may migrate to.
qm cloudinit dump 9000 userDump generated Cloud-Init configPrints the generated user config as a base for a custom file; the other types are network and meta.
qm cloudinit pending VMIDShow pending Cloud-Init changesShows the current values next to those still waiting.
qm cloudinit update VMIDRegenerate the Cloud-Init drive
qm guest cmd VMID COMMANDRun a guest agent commandCommands include ping, fsfreeze-freeze, fstrim, get-osinfo, network-get-interfaces and shutdown.
qm guest exec VMIDRun a program inside a VMGoes through the QEMU guest agent and waits up to 30 seconds for the result by default.
qm guest exec-status VMID PIDCheck a guest agent processReports on a process that an earlier guest exec call returned a PID for.
qm guest passwd VMID USERNAMESet a guest user's passwordAdd --crypted when the password has already been run through crypt().

Virtual machine disks 10

The older one-word forms qm importdisk, qm resize, qm rescan, qm unlink and qm move-disk are aliases of the qm disk subcommands.

CommandWhat it does
qm disk import VMID SOURCE STORAGEImport a disk imageThe image arrives as an unused disk; any format qemu-img reads is accepted.
qm set 9000 --scsi0 local-lvm:0,import-from=/path/to/bionic-server-cloudimg-amd64.imgImport an image as a driveCopies the downloaded cloud image to local-lvm and attaches it as scsi0 in one step.
qm disk move VMID DISK STORAGEMove a disk to other storageThe original stays behind as an unused disk unless --delete is given.
qm move-disk 300 scsi0 other-storageMove scsi0 to another storagemove-disk is the alias the guide uses for qm disk move.
qm move-disk 300 scsi1 --target-vmid 400 --target-disk scsi3Reassign a disk to another VMTakes scsi1 away from VM 300 and attaches it to VM 400 as scsi3, renaming the image for its new owner.
qm disk resize VMID DISK SIZEGrow a VM diskA leading + adds to the current size, a plain value sets it; shrinking is not supported.
qm disk rescanRescan storages for VM disksRefreshes disk sizes and picks up unused disk images; --dryrun writes nothing to the VM configs.
qm disk unlink VMID --idlist STRINGDetach or delete VM disksDisks become unused entries in the configuration; only --force removes them physically.
qm importovf VMID MANIFEST STORAGECreate a VM from OVFTakes cores, memory and name from the manifest and imports its disks; the network is configured by hand afterwards.
qm import VMID SOURCE --storage STRINGImport a guest from another hypervisorThe source is a volume on an import storage, ESXi for example.

Snapshots, clones and migration 9

CommandWhat it does
qm snapshot VMID SNAPNAMETake a VM snapshotAdd --vmstate to include the running state (RAM) as well.
qm listsnapshot VMIDList VM snapshots
qm rollback VMID SNAPNAMERoll back to a snapshotThe VM starts afterwards only with --start, or by itself when the snapshot includes RAM.
qm delsnapshot VMID SNAPNAMEDelete a VM snapshot--force drops it from the config file even if removing the disk snapshots fails.
qm template VMIDConvert a VM into a templateLinked clones can then be deployed from it, much faster than full copies.
qm clone VMID NEWIDClone a VM or templateA normal VM is always copied in full; a template gives a linked clone unless --full is set.
qm clone 9000 123 --name ubuntu2Clone a template with a nameDeploys VM 123 from template 9000 and names it ubuntu2.
qm migrate VMID TARGETMigrate a VM to another nodeStarts a migration task; a running VM moves live only with --online.
qm migrate 106 tre --online --migration_network 10.1.2.0/24Live-migrate over a chosen networkRoutes the traffic of this single migration through the 10.1.2.0/24 network.

Container lifecycle 19

CommandWhat it does
pct listList containersCovers the containers of the node the command runs on.
pct status VMIDShow container status
pct create VMID OSTEMPLATECreate a containerThe second argument is an OS template or a backup file.
pct create 999 local:vztmpl/debian-10.0-standard_10.0-1_amd64.tar.gzCreate a container from a templateThe template is named by its volume ID, exactly as the template list of the storage prints it.
pct start VMIDStart a container
pct start CTID --debugStart with debug outputCollects very verbose log output when a container refuses to start.
pct shutdown VMIDShut down a container cleanlyWaits up to 60 seconds by default; --forceStop makes sure it ends.
pct stop VMIDStop a container immediatelyEvery process inside is ended abruptly, without a clean shutdown.
pct reboot VMIDReboot a containerA shutdown followed by a start, which applies pending changes.
pct unlock VMIDRemove a container lockFor a lock that a migration, snapshot or backup left behind, for instance after a power failure.
pct destroy VMIDDelete a containerIts files are deleted with it; --force destroys a container that is still running.
pct destroy 100 --purgeDelete a container from all configurationsAlso takes the container out of replication jobs, backup jobs and HA.
pct template VMIDConvert a container to a template
pct clone VMID NEWIDClone a containerTemplates give linked clones by default; --full forces a complete copy.
pct migrate VMID TARGETMigrate a containerWorks offline; with --restart a running container is shut down, moved and started on the target node.
pct snapshot VMID SNAPNAMETake a container snapshot
pct listsnapshot VMIDList container snapshots
pct rollback VMID SNAPNAMERoll a container back--start boots the container once the rollback has succeeded.
pct delsnapshot VMID SNAPNAMEDelete a container snapshot

Container configuration and files 22

CommandWhat it does
pct config VMIDShow container configuration
pct pending VMIDShow pending container changes
pct set VMIDChange container options
pct set 100 -memory 512Change container memoryLowers the RAM of container 100 to 512 MB.
pct set 100 -net0 name=eth0,bridge=vmbr0,ip=192.168.15.147/24,gw=192.168.15.1Add a network interfaceCreates eth0 on host bridge vmbr0 with a static address and gateway, even while the container runs.
pct set CTID -onboot 1Start a container at host bootSame effect as the Start at boot checkbox of the web interface.
pct set 100 -mp0 thin1:10,mp=/path/in/containerAdd a storage-backed mount pointAllocates a new 10 GB volume on storage thin1 and mounts it at the given path.
pct set 100 -mp0 /mnt/bindmounts/shared,mp=/sharedAdd a bind mountMakes a host directory visible inside the container at /shared.
pct enter VMIDOpen a shell in a containerEnters the LXC namespace as root, without a login prompt.
pct console VMIDOpen a container consoleA login session through getty; the escape sequence prefix defaults to ^a.
pct exec VMIDRun a command in a container
pct push VMID FILE DESTINATIONCopy a file into a container--perms, --user and --group set mode and owner of the copy.
pct pull VMID PATH DESTINATIONCopy a file from a container
pct df VMIDShow container disk usage
pct cpusetsShow assigned CPU setsLists the host CPUs each running container has been assigned.
pct resize VMID DISK SIZEGrow a container volumeWith a + the value is added to the volume, without it the value is absolute.
pct move-volume VMID VOLUME STORAGEMove a container volumeFor the root filesystem or a mount point; the source stays as an unused volume unless --delete is set.
pct rescanRescan storages for volumes
pct mount VMIDMount container filesystem on hostFor emergency maintenance: the container is locked and can only be started or stopped meanwhile.
pct unmount VMIDUnmount container filesystem
pct fsck VMIDCheck a container filesystem--device picks the volume, and --force checks even when it looks clean.
pct fstrim VMIDTrim container volumesBind mounts and read-only mount points are left out.

Container templates 6

CommandWhat it does
pveam updateRefresh the template listThe list is also refreshed daily by the pve-daily-update timer.
pveam availableList downloadable templates
pveam available --section systemList basic system imagesRestricts the long list to one section.
pveam download STORAGE TEMPLATEDownload a templateOn a cluster, a shared storage lets every node reach the image.
pveam list STORAGEList downloaded templatesPrints full volume IDs, which most other commands accept.
pveam remove TEMPLATE_PATHDelete a template

Backup and restore 17

CommandWhat it does
vzdump VMIDBack up a guestTakes VM and container IDs alike; snapshot is the default mode.
vzdump 777Back up one guest with defaultsThe archive lands in the standard dump directory, normally /var/lib/vz/dump/.
vzdump 777 --mode suspendBack up in suspend modeRelies on rsync plus suspend and resume, which keeps downtime minimal.
vzdump 777 --dumpdir /mnt/backup --mode snapshotSnapshot backup to a directorySnapshot mode needs no downtime; the archive goes to /mnt/backup.
vzdump --all --mode suspend --mailto root --mailto adminBack up all guests with mailBecause mailto is set, the result is mailed through sendmail rather than the notification system.
vzdump 101 102 103 --mailto rootBack up several guestsGuest IDs are listed one after another.
vzdump --mode suspend --exclude 101,102Back up all but some guestsThe exclude option implies --all, so every other guest is backed up.
vzdump 777 --exclude-path /tmp/ --exclude-path '/var/foo*'Skip paths in a container backupShell globs are allowed; a leading / anchors the path to the root of the container.
vzdump 777 --prune-backups keep-last=3,keep-daily=13,keep-yearly=9Back up with retention rulesOlder backups of the guest are pruned by the keep options, written as one comma-separated list.
vzdump 123 --fleecing enabled=1,storage=local-lvmBack up a VM with fleecingOld data is cached in a fleecing image on local-lvm, which helps guest IO during the backup.
vzdump 123 --storage pbs-storage --pbs-change-detection-mode metadataContainer backup by metadata detectionOn Proxmox Backup Server, unchanged files are found through the metadata archive of the previous snapshot.
vzdump 101 --stdout | pct restore --rootfs 4 300 -Clone a container through a pipeStreams the backup of container 101 straight into container 300, created with a 4 GB root volume.
qmrestore ARCHIVE VMIDRestore a VM backupPass - as the archive to read from standard input; --force overwrites an existing VM.
pct restore VMID OSTEMPLATERestore a container backupThe backup file takes the place of the OS template argument.
pvesm prune-backups STORAGEPrune old backupsFalls back to the retention settings of the storage; --dry-run only shows what would go.
pvesm extractconfig VOLUMEShow the config inside a backupPrints the guest configuration stored in a vzdump archive.
pvesm set STORAGEID --bwlimit restore=KIBsLimit restore bandwidthSets a default restore limit for one storage; a bwlimit of 0 on a single restore job lifts it.

Storage 29

CommandWhat it does
pvesm statusShow storage statusOne line per datastore; --content and --enabled filter the list.
pvesm list STORAGEList storage contents
pvesm list STORAGE_ID --vmid VMIDList volumes of one guest
pvesm list STORAGE_ID --content isoList ISO images
pvesm add TYPE STORAGEAdd a storageTypes include dir, nfs, cifs, lvm, lvmthin, zfspool, btrfs, iscsi, rbd, cephfs and pbs.
pvesm add dir STORAGE_ID --path PATHAdd a directory storage
pvesm add nfs STORAGE_ID --path PATH --server SERVER --export EXPORTAdd an NFS share--path is the local mount point, --export the path on the NFS server.
pvesm add lvm STORAGE_ID --vgname VGNAMEAdd an LVM volume group
pvesm add iscsi STORAGE_ID --portal HOST[:PORT] --target TARGETAdd an iSCSI target
pvesm add cifs STORAGENAME --server SERVER --share SHAREAdd a CIFS shareCredentials follow as optional --username and --password; the share becomes available cluster-wide.
pvesm add pbs ID --server SERVER --datastore DATASTORE --username USERNAME --fingerprint 00:B4:... --passwordAdd a Proxmox Backup Server datastoreMakes one datastore of the backup server a storage for the whole cluster.
pvesm add zfspool encrypted_zfs -pool tank/encrypted_dataAdd a ZFS dataset as storageGuest volumes created on it are encrypted with the key material of the parent dataset.
pvesm add rbd NAME --monhost "10.1.1.20 10.1.1.21 10.1.1.22" --pool ceph-external --content images --username pve-rbd --keyring /root/rbd.keyringAdd an external Ceph RBD poolThe keyring comes from the external cluster and should be deleted from /root after the command succeeds.
pvesm set STORAGEChange storage settings
pvesm set STORAGE_ID --disable 1Disable a storage
pvesm set STORAGE_ID --disable 0Enable a storage
pvesm set STORAGE_ID --shared 1Mark a storage as shared
pvesm set STORAGE_ID --content isoSet allowed content typesAfter this the storage accepts ISO images only.
pvesm remove STORAGERemove a storage definitionOnly the definition goes: no data is deleted and nothing is unmounted.
pvesm alloc STORAGE VMID FILENAME SIZEAllocate a volumeSize counts in kilobytes unless it carries an M or G suffix.
pvesm free VOLUMEDelete a volume
pvesm path VOLUMEShow a volume's file system path
pvesm scan nfs SERVERList NFS exports of a server
pvesm scan cifs SERVERList CIFS shares of a server
pvesm scan iscsi PORTALList iSCSI targets of a portal
pvesm scan lvmList LVM volume groups
pvesm scan lvmthin VGList LVM thin pools
pvesm scan zfsList ZFS pools
pvesm scan pbs SERVER USERNAME --password STRINGList Proxmox Backup Server datastores

Cluster 14

CommandWhat it does
pvecm statusShow cluster statusReports the cluster as this node sees it; use it to check a new cluster or a join.
pvecm nodesList cluster nodes
pvecm create CLUSTERNAMECreate a clusterRun on the first node; without link options the local IP address becomes link0.
pvecm create test --link0 10.10.10.1Cluster on a dedicated networkAll cluster communication then uses the interface that holds this address.
pvecm create CLUSTERNAME --link0 10.10.10.1,priority=15 --link1 10.20.20.1,priority=20Create a cluster with two linksRedundant Corosync links, each with a priority that decides which one is used.
pvecm add HOSTNAMEJoin a node to a clusterRun on the joining node and name an existing member; an IP address is recommended.
pvecm add IP-ADDRESS-CLUSTER --link0 LOCAL-IP-ADDRESS-LINK0Join over a separate cluster networklink0 gives the address of the joining node on that network.
pvecm delnode NODERemove a cluster nodeRun from a remaining node after the node being removed has been powered off for good.
pvecm expected EXPECTEDSet the expected votes
pvecm expected 1Regain quorum on one nodeA workaround when quorum is lost, for example before repeating a failed delnode.
pvecm qdevice setup ADDRESSSet up a QDeviceNeeds corosync-qnetd on the external server and corosync-qdevice on every cluster node beforehand.
pvecm qdevice removeRemove the QDevice
pvecm updatecertsRegenerate node certificatesAlso the fix for SSH host key errors after a node rejoins with the same IP or hostname.
pvecm updatecerts --unmerge-known-hostsUnmerge legacy SSH known hostsRemoves the old merged known_hosts symlink on systems that still have it.

Ceph 28

CommandWhat it does
pveceph installInstall Ceph packagesRun on each node; --repository and --version choose the package source and Ceph release.
pveceph initCreate the initial Ceph configurationSafe to call again: an existing [global] section keeps its fsid, auth and pool defaults.
pveceph init --network 10.10.10.0/24Initialise Ceph on a dedicated networkRun on one node only; the configuration file is distributed to the others automatically.
pveceph statusShow Ceph status
pveceph mon createCreate a Ceph monitorRun on each node that should host one; three are recommended, and the first also gets a manager.
pveceph mon destroy MONIDRemove a Ceph monitorThe cluster's only remaining monitor cannot be removed this way.
pveceph mgr createCreate a Ceph managerSeveral managers can be installed, yet a single one is active at a time.
pveceph mgr destroy IDRemove a Ceph manager
pveceph osd create DEVCreate an OSD on a diskA disk used before has to be zapped first.
pveceph osd create /dev/sd[X] -db_dev /dev/sd[Y] -wal_dev /dev/sd[Z]OSD with separate DB and WALWithout -wal_dev the WAL ends up together with the DB.
pveceph osd destroy OSDIDDestroy an OSD--cleanup also destroys the logical volumes underneath.
pveceph osd details OSDIDShow OSD details
pveceph stopStop Ceph servicesActs on ceph.target by default; --service narrows it to one daemon.
pveceph stop --service mds.NAMEStop one Ceph serviceThe service name is a daemon type and its ID joined by a dot.
pveceph startStart Ceph services
pveceph pool lsList Ceph pools
pveceph pool get NAMEShow one pool's status
pveceph pool create NAMECreate a Ceph poolWithout options the pool gets 128 PGs, 3 replicas and a min_size of 2.
pveceph pool create POOL-NAME --add_storagesCreate a pool plus storage entryDefines a Proxmox VE storage for the new pool in the same step.
pveceph pool create POOL-NAME --erasure-coding k=2,m=1Create an erasure-coded poolResults in a data pool and a replicated metadata pool; with k=2 and m=1 it survives one failed OSD.
pveceph pool set NAMEChange pool settingsAccepts options such as --size, --min_size, --pg_num and --pg_autoscale_mode.
pveceph pool destroy NAMEDestroy a Ceph poolDeletion continues in the background; add the remove_storages option to drop the storage entry too.
pveceph mds createCreate a metadata serverCephFS needs at least one MDS configured and running.
pveceph mds destroy NAMERemove a metadata server
pveceph fs create --pg_num 128 --add-storageCreate a CephFS with storageThe data pool gets 128 placement groups and the metadata pool a quarter of that.
pveceph fs destroy NAMEDestroy a CephFSRefused while an enabled cephfs storage entry still points to the file system.
pveceph fs destroy NAME --remove-storages --remove-poolsDestroy a CephFS and its poolsRemoves the underlying Ceph pools and the storage entries in the same run.
pveceph purgeRemove Ceph data and configuration--logs and --crash additionally delete the Ceph logs and crash logs.

High availability 25

HA groups (ha-manager groupadd, groupset, groupremove, groupconfig) are deprecated in favor of HA rules and are not listed here.

CommandWhat it does
ha-manager statusShow HA statusShows the manager and the actual state of each resource; --verbose adds the full CRM and LRM status.
ha-manager configList HA resourcesPrints the resource configuration, optionally only for --type ct or vm.
ha-manager add SIDPut a guest under HASID is a type and an ID such as vm:100 or ct:100; the requested state defaults to started.
ha-manager add vm:501 --state started --max_relocate 2Add a VM with relocate limitAllows two relocation attempts when the VM fails to start.
ha-manager set SIDChange an HA resource
ha-manager set vm:100 --state stoppedAsk HA to stop a resourceChanges the requested state, so the HA stack stops the guest and keeps it stopped.
ha-manager set vm:100 --state disabledDisable a failed resourceThe only way out of the error state, in which HA no longer touches the service.
ha-manager remove SIDRemove a guest from HABy default the resource also leaves every HA rule that references it.
ha-manager migrate vm:100 node2Migrate an HA resourceAn online migration; short form of the crm-command of the same name.
ha-manager relocate vm:100 node2Relocate an HA resourceStops the guest and restarts it on the new node, sometimes faster than moving memory live.
ha-manager crm-command stop SID TIMEOUTRequest an HA stopA timeout of 0 performs a hard stop.
ha-manager rules listList HA rules
ha-manager rules add TYPE RULECreate an HA ruleTYPE is either node-affinity or resource-affinity.
ha-manager rules add node-affinity ha-rule-vm100 --resources vm:100 --nodes node1Prefer a node for a resourcevm:100 prefers node1 but may still run elsewhere.
ha-manager rules set node-affinity ha-rule-vm100 --strict 1Restrict a resource to its nodesA strict rule keeps vm:100 on node1 only instead of merely preferring it.
ha-manager rules add node-affinity ha-rule-negative --affinity negative --resources ct:200,vm:300 --nodes node3Keep resources off a nodeNegative affinity names the nodes to avoid, handy in large clusters.
ha-manager rules add resource-affinity keep-together --affinity positive --resources vm:100,vm:200Keep resources on the same nodeSuits guests that talk to each other a lot, such as a web and a database server.
ha-manager rules add resource-affinity keep-separate --affinity negative --resources vm:200,ct:300Keep resources on separate nodesSpreads heavy guests; resource affinity rules are strict by default.
ha-manager rules set TYPE RULEChange an HA rule
ha-manager rules remove RULEDelete an HA rule
ha-manager crm-command node-maintenance enable NODEPut a node in maintenance modeHA-managed services migrate to other nodes; the command can be submitted from any node.
ha-manager crm-command node-maintenance disable NODEEnd node maintenance modeThe node becomes available again; the mode does not end by itself on reboot.
ha-manager crm-command disarm-ha freezeDisarm HA and freeze resourcesReleases all watchdogs for cluster maintenance; services keep their state and HA ignores failures.
ha-manager crm-command disarm-ha ignoreDisarm HA for manual controlResources leave HA tracking, so they can be started, stopped or migrated by hand meanwhile.
ha-manager crm-command arm-haRe-arm HA after maintenance

Storage replication 12

CommandWhat it does
pvesr listList replication jobs
pvesr statusShow replication statusCovers the jobs of this node; --guest filters by guest ID.
pvesr read IDShow a job's configuration
pvesr create-local-job ID TARGETCreate a replication jobThe job ID is the guest ID and a job number joined by a hyphen, such as 100-0.
pvesr create-local-job 100-0 pve1 --schedule "*/5" --rate 10Replicate every five minutesReplicates guest 100 to node pve1 with bandwidth capped at 10 megabytes per second.
pvesr update IDChange a replication job
pvesr update 100-0 --schedule '*/00'Replicate once per hour
pvesr disable IDDisable a replication job
pvesr enable IDEnable a replication job
pvesr schedule-now IDSchedule a job to run now
pvesr delete IDRemove a replication jobThe job is marked for removal; --keep leaves the replicated data on the target.
pvesr runRun replication jobs nowNormally called by a systemd timer; --id limits the run to one job.

Users and permissions 45

CommandWhat it does
pveum user listList users--full adds group and token information to the list.
pveum user add USERIDCreate a userThe ID is written in the name@realm format.
pveum user add developer1@pve -group developers -passwordCreate a user in a groupThe bare -password option prompts for a password.
pveum passwd USERIDSet a user's passwordNot every realm supports changing passwords this way.
pveum user modify USERIDChange a user
pveum user modify testuser@pve -enable 0Disable a user
pveum user modify testuser@pve -group adminAdd a user to a group
pveum user delete USERIDDelete a user
pveum user permissions joe@pveShow effective permissionsDumps the whole permission tree of the user; --path limits it to one path.
pveum group listList groups
pveum group add GROUPIDCreate a group
pveum group modify GROUPIDChange a group
pveum group delete GROUPIDDelete a group
pveum role listList roles
pveum role add ROLEIDCreate a role
pveum role add PVE_Power-only -privs "VM.PowerMgmt VM.Console"Create a role from privilegesThe privilege names go into one quoted, space-separated list.
pveum role modify ROLEIDChange a role
pveum role delete ROLEIDDelete a role
pveum acl listList access control entries
pveum acl modify PATH --roles STRINGGrant roles on a pathSubjects are given with --users, --groups or --tokens; permissions propagate to child paths by default.
pveum acl modify / -group admin -role AdministratorMake a group administratorsGrants the Administrator role on the root path to every member of admin.
pveum acl modify / -user joe@pve -role PVEAuditorLet a user see everything
pveum acl modify /access -user joe@pve -role PVEUserAdminDelegate user managementOn /access the PVEUserAdmin role covers all users; narrower paths limit it to a realm or group.
pveum acl modify /pool/dev-pool/ -group developers -role PVEAdminGive a group a resource pool
pveum acl modify /vms -token 'joe@pve!monitoring' -role PVEAuditorGrant a role to a tokenA token is addressed as user, exclamation mark, token name.
pveum acl delete PATH --roles STRINGRevoke roles on a path
pveum user token list USERIDList a user's API tokens
pveum user token add USERID TOKENIDCreate an API tokenPrints the token value once; store it, because it cannot be read again.
pveum user token add joe@pve monitoring -privsep 1Create a privilege-separated tokenThe token gets only what its own ACL entries grant, not the full rights of the user.
pveum user token modify USERID TOKENIDChange an API token
pveum user token delete USERID TOKENIDDelete an API token
pveum user token permissions joe@pve monitoringShow token permissions
pveum user tfa listList two-factor entries
pveum user tfa delete USERIDDelete two-factor entriesRemoves every entry of the user unless --id names a single one.
pveum user tfa unlock joe@pveUnlock two-factor authenticationLifts the lockout that follows too many failed second-factor attempts.
pveum realm listList authentication realms
pveum realm add REALM --type STRINGAdd an authentication realm
pveum realm add myrealm1 --type openid --issuer-url https://accounts.google.com --client-id XXXX --client-key YYYY --username-claim emailAdd an OpenID Connect realmExample for Google; client ID and key come from the Google OpenID settings.
pveum realm modify REALMChange realm settings
pveum realm sync REALMSync users from LDAPImports users and groups of an LDAP or Active Directory realm; --dry-run writes nothing.
pveum realm delete REALMDelete a realm
pveum pool listList resource pools
pveum pool add POOLIDCreate a resource pool
pveum pool modify POOLIDChange pool members--vms and --storage add guests and storages; together with --delete they are removed instead.
pveum pool delete POOLIDDelete a resource pool

Node management and updates 30

CommandWhat it does
pvenode task listList finished tasksShows 50 tasks unless --limit says otherwise.
pvenode task list --errors --vmid 100List failed tasks of one guestHelpful for failed backup jobs and similar troubleshooting.
pvenode task log UPIDPrint a task logThe UPID is the unique task ID shown in the task list.
pvenode task status UPIDShow a task's status
pvenode startallStart all guests on the nodeOnly guests set to start on boot are started unless --force is given.
pvenode startall --vms 100,101,102 --forceStart selected guestsStarts the three VMs whether or not they have onboot set.
pvenode stopallStop all guests on the nodeTries a clean shutdown first, then force-stops what is still running after 3 minutes.
pvenode migrateall TARGETMigrate all guests awayUseful before an upgrade; --vms narrows it to chosen guests.
pvenode migrateall pve2 --vms 100,101,102 --with-local-disksMigrate selected guests liveMoves three VMs to node pve2 with live migration of local disks enabled.
pvenode config getShow node options
pvenode config setSet node options
pvenode config set --startall-onboot-delay 10Delay guest autostart after bootWaits 10 seconds after the host boots before the first autostart guest, for slow external resources.
pvenode config set --ballooning-target 90Set the ballooning RAM targetRaises the host memory usage target for automatic ballooning from the default 80% to 90%.
pvenode wakeonlan NODEWake a node over the networkBroadcasts a magic packet with the MAC address stored for that node.
pvesubscription getShow subscription status
pvesubscription set KEYSet the subscription key
pvesubscription updateRefresh subscription status--force contacts the server even while the local cache is valid.
pvesubscription deleteRemove the subscription key
pveperfBenchmark CPU and diskMeasures the disk mounted at the path given, / by default, and reports figures such as fsyncs per second.
apt updateRefresh package listsFirst step of every update; go on only when it finishes without errors.
apt full-upgradeInstall available updatesAlso how a point release is installed; may remove packages to satisfy dependencies.
proxmox-boot-tool statusShow configured ESPsLists the EFI system partitions kept in sync and their state.
proxmox-boot-tool refreshSync kernels to all ESPsCopies and configures all bootable kernels on every ESP registered for syncing.
proxmox-boot-tool format /dev/sda2Format a partition as ESPErases the partition, so double-check the device, for example after replacing a boot disk.
proxmox-boot-tool init /dev/sda2Add an ESP to kernel syncingThe init command also refreshes all configured ESPs.
proxmox-boot-tool kernel listList kernels selected for booting
proxmox-boot-tool kernel pin 5.15.30-1-pvePin a kernel versionBoots this version permanently until the pin is removed; a workaround for hardware incompatibilities.
proxmox-boot-tool kernel pin 5.15.30-1-pve --next-bootPin a kernel for one bootGood for testing whether an updated kernel has fixed an issue.
proxmox-boot-tool kernel unpinRemove the kernel pin
pve-network-interface-pinning generatePin network interface namesWrites .link files so that physical interfaces get stable names nic1, nic2 and so on; needs a reboot.

Certificates 19

CommandWhat it does
pvenode cert infoShow node certificates
pvenode cert set CERTIFICATES KEYInstall a custom certificateTakes the PEM certificate chain and private key; --restart restarts pveproxy afterwards.
pvenode cert deleteRemove the custom certificate
pvenode acme account register default [email protected]Register an ACME accountThe address becomes the contact for renewal notices; default is the account name.
pvenode acme account listList ACME accounts
pvenode acme account infoShow an ACME account
pvenode acme account updateUpdate an ACME accountCalled without new account data it simply refreshes the account.
pvenode acme account deactivate defaultDeactivate an ACME accountOne way to move from the staging directory to production: deactivate, then register again.
pvenode config set --acme domains=example.invalidSet the certificate domain
pvenode acme cert orderOrder a certificateRequests a certificate for the domains set in the node configuration.
pvenode acme cert renewRenew the certificate--force renews although expiry is more than 30 days away.
pvenode acme cert revokeRevoke the certificate
pvenode acme plugin add TYPE IDAdd an ACME plugin
pvenode acme plugin add dns example_plugin --api ovh --data /path/to/api_tokenAdd a DNS challenge pluginThe data file holds the API credentials of the DNS provider, here OVH.
pvenode acme plugin listList ACME plugins
pvenode acme plugin config IDShow an ACME plugin
pvenode acme plugin set IDChange an ACME plugin
pvenode acme plugin remove IDDelete an ACME plugin
pvenode config set -acmedomain0 example.proxmox.com,plugin=example_pluginBind a domain to a pluginOrder the certificate afterwards so the domain is validated through that plugin.

Firewall 7

CommandWhat it does
pve-firewall statusShow firewall statusReads and compiles all rules, so configuration errors show up as warnings.
pve-firewall startStart the firewall service
pve-firewall stopStop the firewall serviceRemoves all Proxmox VE iptables rules, which can leave the host unprotected.
pve-firewall restartRestart the firewall service
pve-firewall compilePrint the compiled rulesShows the generated rule set without applying anything, for testing.
pve-firewall localnetShow the local network aliasPrints the values assigned to the automatically defined local_network alias.
pve-firewall simulateSimulate a connectionDescribe the packet with --from, --to, --dport and --protocol; routing between zones is assumed to work.

API shell 13

Commands whose synopsis ends in [FORMAT_OPTIONS] accept the output options; --quiet suppresses the output completely.

CommandWhat it does
pvesh get API_PATHCall GET on an API pathInvokes the API function directly, without the HTTPS server; only root may do so.
pvesh get /nodesList cluster nodes through the API
pvesh ls API_PATHList child objects of a path
pvesh usage API_PATHShow API usage for a path--command picks create, delete, get or set, and --returns adds the schema of the returned data.
pvesh usage cluster/options -vList datacenter optionsVerbose usage output for the datacenter options path.
pvesh set API_PATHCall PUT on an API path
pvesh set cluster/options -console html5Set the default consoleMakes the HTML5 noVNC console the datacenter default.
pvesh set /cluster/options --tag-style color-map=example:000000:FFFFFFSet a tag colourOverrides the colour that would otherwise be derived from the tag text.
pvesh create API_PATHCall POST on an API path
pvesh delete API_PATHCall DELETE on an API path
pvesh get /nodes/{nodename}/hardware/pci --pci-class-blacklist ""List PCI devices of a nodeShows IOMMU groups, to check that a passthrough device sits in a group of its own.
--output-format jsonChoose the output formatFor commands marked FORMAT_OPTIONS; the values are text (default), json, json-pretty and yaml.
--noheaderHide column headersApplies to the text format only.

Notes

VMs and containers

qm create VMID and pct create VMID OSTEMPLATE make a guest from options; qm clone VMID NEWID and pct clone VMID NEWID copy an existing one, and from a template the result is a linked clone unless --full is set. qm config VMID and pct config VMID print the configuration. qm guest exec VMID runs a program inside a VM through the QEMU guest agent, and pct enter VMID or pct exec VMID does the same for a container. qm destroy VMID and pct destroy VMID delete the guest together with its volumes, so the ID is worth double-checking; add --purge to remove it from backup jobs, replication jobs and HA as well.

Storage and disks

pvesm status lists every storage and pvesm list STORAGE the volumes on one. qm disk import VMID SOURCE STORAGE brings an external image in as an unused disk, qm disk resize VMID DISK SIZE grows a disk (shrinking is not supported), and qm disk move VMID DISK STORAGE and pct move-volume VMID VOLUME STORAGE shift a volume to another storage, keeping the original as an unused volume unless --delete is given. vzdump VMID creates a backup, in snapshot mode by default; pvesm prune-backups STORAGE applies the retention settings of the storage to existing backups.

Cluster, HA and the API

pvecm status and pvecm nodes show the cluster as the local node sees it, and pvecm expected 1 is the documented workaround when a remaining node has lost quorum. qm migrate VMID TARGET and pct migrate VMID TARGET move a guest to another node; a running container needs the --restart option, because containers cannot be migrated live. ha-manager status shows HA resource state and ha-manager add SID puts a guest under HA, after which the normal start and stop commands are forwarded to the HA stack. pvesh get API_PATH calls any API path from the shell as root. pveam available lists container templates and pveam download STORAGE TEMPLATE fetches one.

Questions

What is the difference between qm and pct?

qm manages QEMU/KVM virtual machines and pct manages LXC containers. Many subcommands carry the same name in both tools, for example start, shutdown, snapshot, clone and migrate.

How do I run a command inside a VM from the host?

Use qm guest exec VMID followed by the command. It goes through the QEMU guest agent in the VM and by default waits up to 30 seconds for the result.

How do I regain quorum when a single node is left?

Run pvecm expected 1 on that node. The Administration Guide gives it as a workaround, for example when pvecm delnode NODE fails because the remaining node has lost quorum; repeat the failed command afterwards.

How do I import a VM disk from another hypervisor?

Copy the image to the host and run qm disk import VMID SOURCE STORAGE; the image is added to the VM as an unused disk, in any format that qemu-img supports. A whole OVF export becomes a new VM with qm importovf VMID MANIFEST STORAGE.

Sources and checks

Every row on this page was matched to a line in the documentation above on . A binding that could not be found there is not listed.

73 rows from the earlier version of this page were removed in that check: duplicates, and bindings the documentation does not list or lists differently.

Found a wrong or missing key? Tell us and it gets checked against the source. How we verify describes the process and its limits.