Do CS. It seems like you’re looking to do more coding than analysis/forecasting.

Nothing about this is novel though; the fact that language models are able to uncover sensitive training information has been a thing for a while now.

Data pre-processing.

The startup I work at is fairly small so we don’t really have anyone who deals with the data itself (e.g., data engineers, data scientists, etc.). That leaves the MLEs to do most of the grunt work.

Well I don’t know your situation but I feel like the “never have time” excuse may not necessarily be true. Even creating a page in Notion and writing down one line is enough for me. I feel like what was holding me back before was the trap of perfectionism. I wouldn’t want to write anything unless I could make it into some conference-poster-quality page.

Don’t be sad, it’s just a part of how things are you just have to choose a method and stick to it.

I personally use Notion. I’ve created a database and added properties like date, venue, authors, organizations, etc.

For example, the other day I needed to recap what the BLIP paper was about so I just searched the paper in the database and took a look at the page. On that page I’ve highlighted different text with different colors depending on when I came back to read it.

Took me a while to get this working and into the habit of it though.

I’m a little curious why this post has so many upvotes. I guess it shows that things really have changed a lot.

Repetition and working with them. I hope you’re not under the impression that reading a paper once is going to help you remember it. I have to read a paper at least 3-4 times before I feel like I actually really understand it.

I remember reading somewhere that people are only able to retain 10-15% of the information they read in the first go or something.

TL;DR The more constraints on the model, the more time should spend analyzing your data and formulating your problem.

I’ll agree with the top comment. I’ve also had to deal with a problem at work where we were trying to perform product name classification for our e-commerce product. The problem was that we couldn’t afford to have anything too large or increase infrastructure costs (i.e., if possible we didn’t want to use any more GPU computing resources than we already were).

It turns out that extensive EDA was what saved us. We were able to come up with a string-matching algorithm sophisticated enough that it achieved high precision with practically no latency concerns. Might not be as flexible as something like BERT but it got the job done.

TL;DR The more constraints on the model, the more time should spend analyzing your data and formulating your problem.

I’ll agree with the top comment. I’ve also had to deal with a problem at work where we were trying to perform product name classification for our e-commerce product. The problem was that we couldn’t afford to have anything too large or increase infrastructure costs (i.e., if possible we didn’t want to use any more GPU computing resources than we already were).

It turns out that extensive EDA was what saved us. We were able to come up with a string-matching algorithm sophisticated enough that it achieved high precision with practically no latency concerns. Might not be as flexible as something like BERT but it got the job done.

The reason why is because most researchers can’t be bothered because no one pays attention to it anyway. I’m always doubtful about the number of researchers who even properly understand statistical testing.

I’d be grateful if a paper ran experiments using 5-10 different random seeds and provided the mean and variance.