I wanted to build my server on AM4, and the cheapest motherboard with 10GbE was 400 dollars(!!). So I spent a bunch of time messing with Thunderbolt instead. Here are some notes I took.
In theory, Thunderbolt as a connectivity standard is great! There are a couple of things that are confusing!
Platform-specific issues
On Linux, devices are not authorized when connected. To "authorize" the connection, you have to set up udev rules. udev is the part of Linux that reacts when you plug a device in, and a udev rule can tell it to mark every new Thunderbolt device as authorized.
However, I found it easier to disable Thunderbolt security in the BIOS. This only leaves you vulnerable to physical attacks, where someone can plug something into your device.
Hot plugging did not work with my Mac. I fixed it by updating the firmware on the Thunderbolt controller chip with fwupdmgr (sudo fwupdmgr get-devices lists it, sudo fwupdmgr update installs the update); your manufacturer's website may have the update too.
After that, on Mac, it just works™.
What's the big deal with Thunderbolt networking?
20 Gb!
Hardware
Opinion: Why are people not considering Thunderbolt as an alternative to 10GbE? 10GbE is still an unusually expensive/premium add-on for desktop computers/Macs, despite its age.
10 Gb can be cheap if you only work off a desktop with 10GbE or SFP+.
If not, you will have to buy a dongle to connect your server. A 10GbE dongle is around 200 dollars: ubiquiti usb-c 10gbe
For my specific scenario, I wanted to build on AM4, as the cost/price performance made the most sense for me. Here's what I bought instead of the $400 board:
- asus pro b550m - 80 USD
- thunderboltex4 add-in - 100 USD
- thunderbolt 4 cable - 15 USD
If you want to learn more about my new server, wait for my next post.
For the add-in card, I would wait to find an open box/return, since many people don't do the research on what motherboards are compatible. There weren't any AM4 microATX boards with built-in USB4; typically high-end ATX boards have it, and I wanted to build in a small case.
Leftovers
My fixation on microATX is because I have a case sitting around that only fits microATX!
Reduce, reuse, recycle.
Debugging tools:
- udevadm monitor
- bolt
- tbtools
I am aware that 10 Gb (gigabit) networking is 1,250 megabytes per second, while most HDDs can only write at 150 MB/s.
On TrueNAS, I was not able to install tbtools. I was in developer mode and I ran the installation script from a data drive. Oh well, I figured it out.
generic TrueNAS is an appliance comment
USB4 != Thunderbolt 4
