Meeting 11/04/2020 – Amran – Scraping web

Overview :

  • Scraping detail page

Challenges :

  • Struktur page tiap halaman berbeda-beda
    • Baca-baca artikel tentang teknik penanganan struktur yang berbeda dengna tag yang sama

Feedback :

Next Steps :

  1. Insert or update MongoDB jobPosting collection from JobPosting object
  2. Wrap crawler into Zeebe worker using zeebe-grpc (Python)
  3. Store HiringOrganization Logo Image to S3+MongoDB
  4. Run Workflow in Zeebe Cloud – with Hendy
  5. Queue Next Job Collection Pages

Similar Posts

Leave a Reply

Your email address will not be published. Required fields are marked *