Oooh! I’ve been waiting for someone to try to replace LaTeX.

I used Zotero all through uni, but I’ve recently discovered JabRef. Zotero probably has more features and polish, but JabRef has one killer feature: Your plaintext BibLaTeX file is your reference database. So you can version control and collaborate on your reference file with git. No export required.

One of the first real programs I wrote was a program to display telemetry data from a CAN bus. I was on the solar car team at uni, and we wanted to be able to view the data from the various systems live during the race. The CAN network was connected to a CAN-ethernet converter, which sent UDP packets over a wireless ethernet link to our lead car. I had no experience with networking, or UDP or CAN at all, but I had some documentation and a lot of free time, so I got to work.

Each device on the CAN network had a bit mask to identify it. For example, the bit mask for the motor controller might have been 0x1200. This meant that any packet starting with 0x12 belonged to the motor controller. For example, 0x1201 was one type of message, and 0x1202 another type, but both belonged to the motor controller.

There was specific logic for each device on the network, so you needed to first figure out which device owned a packet using the bit mask, then apply the relevant logic to decode the packet.

Looking back, I realise the correct way to approach this would be to have a list of bit masks:

0x1200
0x1300
0x1400

Then simply bitwise & any incoming packet with 0xff00, and lookup the result in the list of bit masks.

Not knowing better however, what I actually did was create a giant dictionary of every possible packet value, so I could lookup any packet and determine which system it came from. This was so repetitive that I had to make use of my newfound super-power – vim macros – to complete the 8000 line dictionary…

Excerpt from real code below:

{
    0x102:
    {
        'name':             'SHUNT_CMU_STATUS_TEMPERATURE_AND_VOLTAGE_1_2',
        'data':
        [
            'cell_0_voltage',
            'cell_1_voltage',
            'cell_2_voltage',
            'cell_3_voltage',
        ],
        'unpack_string':    'intle:16, intle:16, intle:16, intle:16'
    },

    0x103:
    {
        'name':             'SHUNT_CMU_STATUS_TEMPERATURE_AND_VOLTAGE_1_3',
        'data':
        [
            'cell_4_voltage',
            'cell_5_voltage',
            'cell_6_voltage',
            'cell_7_voltage',
        ],
        'unpack_string':    'intle:16, intle:16, intle:16, intle:16'
    },
}

Can I suggest duckdb? You can start out writing SQL directly on top of CSV and Parquet files. But then if you want/need to do something more complicated, you can import duckdb into Python, keep running the SQL you already wrote, convert it to a Pandas or Polars dataframe, transform it, then query the result in SQL.