Meeting 11/04/2020 – Amran – Scraping web
Overview :
- Scraping detail page
Challenges :
- Struktur page tiap halaman berbeda-beda
- Baca-baca artikel tentang teknik penanganan struktur yang berbeda dengna tag yang sama
Feedback :
Next Steps :
- Insert or update MongoDB
jobPosting
collection fromJobPosting
object - Wrap crawler into Zeebe worker using zeebe-grpc (Python)
- Store HiringOrganization Logo Image to S3+MongoDB
- Run Workflow in Zeebe Cloud – with Hendy
- Queue Next Job Collection Pages