ePrivacy and GPDR Cookie Consent by Cookie Consent
0 votes

Is there a way to get attention from the analysts to fix this problem of distributing corrupt files on the datamart?


The problem has actually been going on longer than that github issue has been open.

Seems to me to easy enough to mitigate: the process that pushes the files into the queue needs to ensure they are XML valid before pushing them into the datamart.  But of course I am not aware of whatever the complexity of what is going on behind the scenes.

1 Answer

0 votes
I think you've answered your own question. That github project has all the contact info of everyone involved, no?

Unless your question is "how do I get people to do things" in which case I'm not sure this is the best place to get an answer :P
I'm not sure if that github page actually has the the attention of the correct powers that be to make the improvements happen or not.
Well you can see the main contributors on this page:


- which appear to be those commenting in your GitHub issue too. There have been plenty of recent work done on the project too.

I'm not sure "powers that be" is the correct way to think about this. It's a public repository so anyone in the world can submit a merge request. Sure, one of the maintainers would have to review it before merging, but I'm sure that if a merge request were submitted and linked to that issue it would be accepted.

So I'm not sure it's a powers that be issue, more of a prioritization and developer hours issue. Do you have any developers on your team that know how to submit a merge request and can take a crack at fixing the problem?
As far as I know the problem is not actually in sarracenia, so there is nothing to fix there (issue there should probably closed as invalid).  

From my understanding, the problem is somewhere in the process that creates (edits) the xml files, so the issue should be redirected to the people who are in charge of that process.

My question, is if these people are investigating/fixing the issue.   As far as I know there is no public github for that process.  (if there is I would be happy to review what is going on over there).

There is another discussion about the load that 'scrapers' put on the the dd web site, which I know from my perspective I have to scrape things because sarracenia delivers corrupt files (which is not sarracenia's fault, from what I know)

Of course I may be misinformed, so happy to be corrected.
Hmmm, well I can't add much more than that. Good luck :o
Welcome to MSC-Open-Data Q&A where you can ask questions about the open data of the Meteorological Service of Canada and receive answers from other members of the community.

Bienvenue sur MSC-Open-Data Q&A où vous pouvez poser des questions sur les données ouvertes du Service Météorologique du Canada et recevoir des réponses d'autres membres de la communauté.