jamboca

Hello peers,

I am currently on a high visibility project at my company involving collaboration of many data scientists and data engineers in a non-purely engineering domain.

I am a non-senior data engineer supporting this project and i have a lot of thoughts about the direction of it, how things could be improved, and where we are going wrong. however, i don't have the space or title to be heard out. data scientists feel like they see the platform team as an enemy rather than a power multiplier. additionally, the senior platform engineer supporting this project is spread thin across many projects and doesn't have their hands on the data. yet, they are directing us on how to work on this project. i get very annoyed working with these people as i see and know what the project needs, but instead i am just instructed to follow the lead of a senior engineer who is not really in touch with the needs of the project but thinks they are.

currently we use a complex pipeline which i was instructed to build. i am the most in touch with the underlying data and intimately understand the needs on both the platform and DS side despite not receiving super concrete requirements from the side of the "scientists."

i would like to implement some visionary functionality, high level architectural differences in parallel with current development/platform support work, and then one day say something like "hey, i built a copy of what we are doing but faster, better, etc. here is the proof and let me know what you think." however, it would basically automate the jobs of the people on the platform side, making enemies with at least a few people and especially the lead engineer who makes these calls which i disagree with. at least, i would like the stakeholders to see how our current approach is hindering results and how i have a vision for a better one. however, it's not my place or my title to do so.

i want to just show a director or our manager that i can fix this project without doing it in a hostile way. but i don't see how to do this given the organizational structure/conditions. would it be better to leave and go to another company in hopes of a better atmosphere where i can be heard out for my ideas?

Stuck, Data Engineer

hello hacker news

i am starting soon remotely as a junior data engineer at a health tech company. i have some experience in this kind of data stuff but not in the specific technologies of the company.

it's my first real job and i have some questions if anyone more experienced would be able to pitch in?

what should i be on the look out for, and how can i prepare? for my long term career, what are the options? should i stay here a while or move on after a time for more $? im not sure what the roadmaps look like but i would appreciate any advice from people with some time in the world already.

I write my concerns on HN because I cannot find anybody other than my parents who can help me to resolve this general disillusion with university. I am in my first year of STEM studies at an average state school in the USA, feeling extremely bored while still achieving top grades in the classroom and carrying seemingly important extracurriculars outside of it. I did the first year of CS curriculum while in high school, so I am scheduled to graduate in three years.

University is not shaping up to be what I expected. I expected to have the conversations that would shape my life here, but instead I have found empty people with little ambition and little interest in the world outside of a regional microcosm. Where is the inspirational exchange, the creation of friendships for life? (for reference, I felt like I had all of this in high school)

For high school I went to an international boarding school, and one week of conversations there was more interesting than anything of this semester. I am interested in almost every subject at school... I really love learning new things. With the expectation of university as more than just a technical education, I question why I am here. All I have learned this semester is equations from the math book, which I could realistically learn anywhere.

My current ideas are: (1) transfer to a liberal arts college to receive an education in a wide range of more interesting topics (philosophy, history, economics, other things I studied in HS) alongside more interesting and more passionate people, or (2) take a gap semester in Mexico City or Dar es Salaam to make life more interesting while self teaching myself everything that I can learn by being here already. That way, I might get more of the cultural education I had in high school which I now feel is lacking.

I am asking for general life advice. Life went from extremely interesting (surrounded by a global community of people from over 50 countries) to extremely boring and generally meh due to a lack of diversity in thoughts, ambitions, ways of life, etc. Does anybody relate?