add some proxmox docs

Author: laksith19

docs/staff-docs/nix/new-hosts.md

@@ -31,29 +31,75 @@ when writing, go to nix options search, thoroughly look over all options availab
 
 
 ## 2. New VM on Proxmox
-### General
 
 - click create virtual machine
+- Make sure you have the Advanced box checked at the bottom to be able to see all the config options, when
+provisioning a new VM.
+- There is a very helpful "Help" button in the bottom left of the VM creation window that will take you to the
+appropriate page in the Proxmox docs that can provide really helpful information / context.
+- Things keep chaning and new tools and features are introduced fairly often, so if you think something in these
+docs need to be updated talk to a root staffer (the current SMs will be a good place to start).
+
+### General
+
+- Select a Node that has capacity for the about of CPU and RAM resources you wish to allocate.
 - choose VM name from my little pony (check LDAP and run dig to ensure host doesnt already exist!) (TODO link LDAP docs)
-- select start at boot
+- leave the resource pool empty (TODO: reconsider using this feature as more guests are migrated to proxmox)
+- select start at boot (order, delay and timeout can be left as is)
+- Add a Tag for the OS the VM is going to run (nix / debian) and the primary purpose of the VM (webhost, staffvm etc...)
 
 ### OS
 
-- select nix ISO image
+- If you're going to be using a network PXE boot (TODO: set one up this would be real nice), select "Do not use any media".
+Otherwise select the storage where you uploaded / downloaded the ISO and the ISO image you want to boot from.
+- Leave the Type and Version as their default values (Linux / xxx Kernel)
 
 ### System
-
-- graphic card: SPICE
+<!-- TODO @laksith19: Usually prefer SPICE / QXL but it seems like there's a bug in the Driver in the
+current lts kerne (6.12.40) that cause the guest displays to freeze up randomly and prevent reboots without
+using a hypervisor level reset (basically a power-cycle) which is not ideal. As we're probably never going to
+have graphical VMs, the graphics performance is not important and it's probably best to just stick to default.
+Leaving this TODO, in case it's worth re-visiting in the future. -->
+- graphic card: Default
 - machine: q35
 - SCSI Controller: VirtIO SCSI single
 - enable QEMU agent
 - BIOS: OVMF (UEFI)
-- EFI Storage: primary-zfs
+- EFI Storage: vmdata
 - de-select Pre-Enroll keys
+- de-select Add TPM (Unless you really know you need it)
 
 ### Disks
-- enable SSD emulation
-- storage: primary-zfs
+Most of these options are the default values enumarated for completion, **except for the ones in bold**
+- Bus/Device: SCSI 0
+- SCSI Controller: VirtIO SCSI single
+- Cache: Default (No cache) (This just means that the host page cache is not used and the performace will be similar
+to the VM having direct access to the disks. If you know what you're doing and think for the use case of the VM there's
+a better a option the [Performance Tweaks](https://pve.proxmox.com/wiki/Performance_Tweaks#Disk_Cache)
+section of the Proxmox Wiki is a really helpful guide.)
+- **Enable Discard**
+- Enable IO Thread
+- **Storage: vmdata**
+- **Disk size: xxGiB**: This space is thinly provisioned (or sparse) in ZFS and will not be used or reserved for the VM unless
+actually used by the VM. This allows us to be able to overprovision VM disks with additional capacity, but make sure you keep an
+eye the actual usage of the underlying storage pool (vmdata in this case).
+- Format: Raw disk image
+- **Enable SSD emulation**
+- Disable Read-only
+- Enable Backup
+- Disable Skip replication
+- Async IO: Default (io_uring) (TODO: Probably worth changing this to native when we eventually 
+migrate to CEPH - [context](https://forum.proxmox.com/threads/proxmox-ve-7-2-benchmark-aio-native-io_uring-and-iothreads.116755/))
+
+### CPU
+Ignore the advanced settings for this section the defaults are sufficient.
+- Sockets: 1
+- Cores: 2 (a generalized default suggestion that should work for most use cases).
+- Type: x86-64-v2-AES (Select the lowest compatible virtual QEMU CPU type in the cluster, this is important for live
+migrations to work. Technically setting this to host will give you maximum performance but you'll not be able to
+live migrate the VM from one host to another.)
+
+### Memory
 
 ### Other Hardware
 - give at least 32GB mem!