undefined | Better HN

0 pointsnerdponx3y ago0 comments

Can't speak for FAANG, but you can do well at some companies with "ML Developer" or even "Data Engineer" if you have database admin experience and/or are willing to spend time studying up.

0 comments

brutus12133y ago

Just wanted to add: DL Engineer != MLOps engineer. DL Engineers who deal with scientists have a much harder job IMHO as the scientists ask for 10 different things, each taking a lot of effort to implement. The current DL stacks (e.g. Tensorflow/keras) make it very easy to implement the standard stuff. But if you want to go off the path, it is really very hard (I wonder if Pytorch makes that significantly easier if anyone cares to chime in).

nerdponxOP3y ago

Good point in general about working with "scientist code". You definitely will want to at least understand some of the math in order to be effective at implementing it.

I don't think Torch is any easier than Keras if you don't know what you're doing.

I think data engineering is probably the easiest transition for software devs with backend web (read: database and Python) experience. Free learning resources abound nowadays, as well as decent-quality books and online courses. It's largely the same set of engineering "macro" skills (proejct planning, cost-time-correctness tradeoffs, levels of abstraction, deploying stuff to the cloud), and the specific engineering "micro" skills (database performance tuning, indexing, storage and compute cost management, setting up ETL pipelines, integrating with dashboard systems like Tableau) should be well within the capabilities of a mid-senior backend web developer.

The hardest part IMO would be getting up to speed to the stats and data analysis basics enough to be conversant with the analysts and data scientists who would be your primary stakeholders and users. But that too can be done with help from the multitude of free learning resources and chatrooms/forums.

IMO it's worth the effort if you're looking to transition. There is a shortage of good, conscientious data engineers right now. Hiring pipelines are clogged with aspiring junior data scientists who are willing to put up with data engineering for a couple of years to try and get some experience on their resume. Data engineers' skillsets are highly portable across companies and industries, and will remain in demand for many years to come, even as companies find that they can't justify hiring a data scientist, so pay and hiring potential will be accordingly high. And as a data engineer you will quickly establish yourself as an essential team member in all but the most dysfunctional organizations, and your value will be immediately apparent whenever someone wants to build a dashboard or pull a report and it all "just works".

The only thing to beware of is the usual caveat of companies trying to short-sightedly hire inexperienced juniors in mid-senior roles to save a buck, setting both you and them up for failure. But that's a caveat in any field where you're starting from the bottom. If you're already an experienced software engineer, you should hopefully be able to either see through that bullshit, or to be able to handle it somewhat more gracefully than a true junior could.