Software Engineer, Web Crawling
News Break
About NewsBreak
NewsBreak is redefining the way users interact with local news and their communities. By bridging local users, local content creators, and local businesses, our mission is to foster safer, more vibrant, and authentically connected lives. Through robust collaborations with thousands of local publishers and businesses across the nation, NewsBreak is revolutionizing how a new wave of readers access and engage with essential, locally sourced content & information.
Since our inception in 2015, our trajectory has been nothing short of remarkable. We proudly stand as the nation’s premier local news app, boasting 60 million monthly active users and clinching the top spot in the news category on both Google Play and the App Store.
As a profitable, Series-C unicorn startup, our headquarter nestles in the tech hub of Mountain View, California, with other offices in New York City and Seattle. For more information, visit www.newsbreak.com/about
About the Role
We are seeking a talented Backend Engineer to join our team. You will play a crucial role in developing and maintaining content system solutions that collect and process data from various sources. If you are passionate about data extraction, have a deep understanding of web techniques, and enjoy solving complex challenges, we want to hear from you. Your work will boost the success of various projects and contribute to our continued growth and success.
Responsibilities
- Design, develop, and maintain content integration solutions to collect data from various sources.
- Collaborate with data scientists and analysts to define data extraction requirements.
- Monitor and troubleshoot content acquiring to ensure data accuracy and reliability.
- Implement data validation and quality checks to ensure the integrity of collected data.
- Stay up-to-date with industry trends and best practices in web scraping and data extraction.
- Work closely with cross-functional teams to integrate integrated data into our products or services.
- Identify and address related potential legal and ethical considerations.
Requirements
- Bachelor's degree in Computer Science, Information Technology, or a related field.
- Proven experience as a Web Crawler Engineer or similar role.
- Strong proficiency in programming languages such as Python, Java, Golang.
- Knowledge of web scraping libraries and frameworks (e.g., Scrapy, Beautiful Soup).
- Familiarity with web crawling challenges, including handling dynamic content, CAPTCHAs, and IP rotation.
- Understanding of HTML, XPATH, CSS, and JavaScript for parsing web content.
- Experience with database systems for storing and managing collected data.
- Excellent problem-solving skills and attention to detail.
- Strong communication and collaboration skills.
- Knowledge of web scraping ethics and compliance with website terms of service.
Benefits
We offer competitive benefits package:
- Health, dental, and vision care for you and your family
- Top-tier 401(K) plan with company matching
- Paid time off and paid holidays
- Paid parental leave
- FSA and commuter benefits programs
- Team activity budget