Senior Phyton Developer - AI/ML

We are seeking an experienced Senior Python Developer with expertise in web scraping, data crawling, database management, and advanced skills in Artificial Intelligence (AI) and Machine Learning (ML). The ideal candidate will be responsible for building robust, scalable systems that collect, manage, and analyze vast amounts of data from various sources. You will work closely with our data science team, contributing to innovative AI/ML models and optimizing data-driven solutions.

Responsibilities:

1. Web Scraping and Crawling:

• Design, implement, and maintain efficient web scraping and crawling tools to gather structured and unstructured data from a variety of online sources.
• Ensure scraping scripts are optimized for performance, speed, and reliability, while adhering to legal and ethical data scraping practices.
• Develop methods to handle complex anti-scraping mechanisms (CAPTCHAs, rate limits, etc.).

2. Data Pipeline and Integration:

• Build and maintain scalable data pipelines to collect, clean, and process data for analysis and machine learning purposes.
• Integrate data from multiple sources including APIs, databases, and raw files into a unified data framework.
• Ensure data consistency, integrity, and availability across different systems.

3. Database Management:

• Design and maintain databases (SQL and NoSQL) that can efficiently store and retrieve large-scale data for analysis and modeling.
• Optimize database performance, including query optimization and data indexing.
• Manage database migrations, backups, and restorations as needed.

4. AI & Machine Learning:

• Collaborate with data scientists to develop and deploy AI/ML models using Python frameworks like TensorFlow, PyTorch, or Scikit-learn.
• Write efficient and reusable code for model training, validation, and deployment in production environments.
• Continuously monitor and fine-tune models to improve performance and accuracy.

5. Automation and Performance Optimization:

• Automate repetitive data collection and processing tasks to ensure system scalability.
• Optimize code and processes for maximum efficiency, speed, and resource usage.
• Ensure system reliability, including implementing robust error-handling mechanisms.

6. Collaboration and Mentorship:

• Work with cross-functional teams, including data engineers, data scientists, and product managers, to deliver high-quality solutions.
• Mentor and guide junior developers, sharing best practices, reviewing code, and providing technical advice.

7. Documentation and Reporting:

• Maintain clear and comprehensive documentation of all systems, processes, and code.
• Regularly report on project progress, challenges, and outcomes to relevant stakeholders.
• Stay up-to-date with the latest technologies, tools, and best practices in Python development, scraping, AI, and ML.

Qualifications:

• Bachelor's or Master's degree in Computer Science, Information Technology, or a related field.
• 5+ years of experience in Python development, specifically in web scraping, data crawling, and data management.
• Proficiency with Python scraping tools like Scrapy, BeautifulSoup, Selenium, etc.
• Experience working with databases (SQL, NoSQL) and data pipeline tools like Apache Airflow, Apache Kafka, or similar.
• Strong understanding of AI/ML frameworks such as TensorFlow, PyTorch, Scikit-learn, etc.
• Familiarity with cloud platforms (AWS, GCP, or Azure) for deploying AI/ML models.
• Excellent problem-solving and communication skills.
• Ability to work independently and manage multiple projects simultaneously.

This position is ideal for someone who enjoys solving complex data challenges, has a passion for AI/ML, and thrives in a fast-paced, data-driven environment. If you’re excited about building cutting-edge solutions in the field of web scraping and AI, we’d love to meet you!