Skip to content

added loader class and mapping for OEWS data#2

Open
ming-afk wants to merge 1 commit intomainfrom
ming/api-explore
Open

added loader class and mapping for OEWS data#2
ming-afk wants to merge 1 commit intomainfrom
ming/api-explore

Conversation

@ming-afk
Copy link
Copy Markdown

resolves #1 . Scripts included here.

@ming-afk ming-afk requested a review from thomaslin2020 July 28, 2023 19:32
@trafico-bot trafico-bot Bot added the 🔍 Ready for Review Pull Request is not reviewed yet label Jul 28, 2023
@thomaslin2020
Copy link
Copy Markdown
Collaborator

  • Please modify the code so that instead of reading the excel file locally, read it from the url. This also means that we don't need to add too many custom rules in .gitignore.
  • What's the problem with repetitions in OCC_TITLE? Would O_GROUP be important for our use cases?

@ming-afk
Copy link
Copy Markdown
Author

  • with the current method I manually downloaded the excel data. The data updates once a year, would it be unnecessary to read the url as we are good for one year now?
  • The data looks like below. For example the "Chief Executives" position has different O_GROUP but all other data being the same. Case where the O_GROUP and other data are both different is rare. But I'll check the meaning of O_GROUP further to ensure I understand it correctly.
Screen Shot 2023-07-28 at 10 20 25 PM

@thomaslin2020
Copy link
Copy Markdown
Collaborator

I think it's still good to have files downloaded through requests as data files aren't easy to share with github

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

🔍 Ready for Review Pull Request is not reviewed yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Backend] Job and Salary Data Collection

2 participants