ePrivacy and GPDR Cookie Consent by Cookie Consent
0 votes
by
Sometimes download citypage_weather files gets an older version of the file.

If you keep downloading the same URL over and over, sometimes you get an old file, sometimes you get a new file.   Shouldn't eventually all the hosts in the cluster serve the same file (eg they might be out of sync for a few seconds), but not hours later...

I wrote a test script that shows the md5sum keeps flipping back and forth. (time difference between the two files is 1 minute).  The file was posted around 12 hours earlier than when this test is run, so no reason for it to be flipping.  (HEF files are just updated once per day this time of year).

while [ 1 ]
do
echo -n "$(date) "
curl -s  https://dd.weather.gc.ca/citypage_weather/xml/HEF/s0000849_e.xml | md5sum
sleep 5
done
Sat 06 Jun 2020 04:16:33 PM EDT 4d0320f1af9c2992a5a626100856f902  -
Sat 06 Jun 2020 04:16:38 PM EDT 64f60d417fd3b165ec2cd1682198890b  -
Sat 06 Jun 2020 04:16:43 PM EDT 4d0320f1af9c2992a5a626100856f902  -
Sat 06 Jun 2020 04:16:49 PM EDT 64f60d417fd3b165ec2cd1682198890b  -
Sat 06 Jun 2020 04:16:54 PM EDT 64f60d417fd3b165ec2cd1682198890b  -
Sat 06 Jun 2020 04:17:00 PM EDT 64f60d417fd3b165ec2cd1682198890b  -
Sat 06 Jun 2020 04:17:05 PM EDT 4d0320f1af9c2992a5a626100856f902  -
Sat 06 Jun 2020 04:17:10 PM EDT 64f60d417fd3b165ec2cd1682198890b  -
Sat 06 Jun 2020 04:17:15 PM EDT 4d0320f1af9c2992a5a626100856f902  -
Sat 06 Jun 2020 04:17:20 PM EDT 4d0320f1af9c2992a5a626100856f902  -
Sat 06 Jun 2020 04:17:26 PM EDT 4d0320f1af9c2992a5a626100856f902  -
Sat 06 Jun 2020 04:17:31 PM EDT 64f60d417fd3b165ec2cd1682198890b  -
Sat 06 Jun 2020 04:17:36 PM EDT 4d0320f1af9c2992a5a626100856f902  -
Sat 06 Jun 2020 04:17:42 PM EDT 64f60d417fd3b165ec2cd1682198890b  -
Sat 06 Jun 2020 04:17:47 PM EDT 64f60d417fd3b165ec2cd1682198890b  -
Sat 06 Jun 2020 04:17:52 PM EDT 64f60d417fd3b165ec2cd1682198890b  -
Sat 06 Jun 2020 04:17:58 PM EDT 4d0320f1af9c2992a5a626100856f902  -
Sat 06 Jun 2020 04:18:03 PM EDT 64f60d417fd3b165ec2cd1682198890b  -
Sat 06 Jun 2020 04:18:08 PM EDT 4d0320f1af9c2992a5a626100856f902  -
Sat 06 Jun 2020 04:18:13 PM EDT 4d0320f1af9c2992a5a626100856f902  -
Sat 06 Jun 2020 04:18:19 PM EDT 4d0320f1af9c2992a5a626100856f902  -
Sat 06 Jun 2020 04:18:24 PM EDT 64f60d417fd3b165ec2cd1682198890b  -
Sat 06 Jun 2020 04:18:29 PM EDT 64f60d417fd3b165ec2cd1682198890b  -
Sat 06 Jun 2020 04:18:35 PM EDT 64f60d417fd3b165ec2cd1682198890b  -
Sat 06 Jun 2020 04:18:40 PM EDT 64f60d417fd3b165ec2cd1682198890b  -
Sat 06 Jun 2020 04:18:45 PM EDT 4d0320f1af9c2992a5a626100856f902  -
Sat 06 Jun 2020 04:18:50 PM EDT 4d0320f1af9c2992a5a626100856f902  -
Sat 06 Jun 2020 04:18:56 PM EDT 64f60d417fd3b165ec2cd1682198890b  -
Sat 06 Jun 2020 04:19:01 PM EDT 64f60d417fd3b165ec2cd1682198890b  -
Sat 06 Jun 2020 04:19:06 PM EDT 4d0320f1af9c2992a5a626100856f902  -
Sat 06 Jun 2020 04:19:11 PM EDT 64f60d417fd3b165ec2cd1682198890b  -
Sat 06 Jun 2020 04:19:17 PM EDT 4d0320f1af9c2992a5a626100856f902  -
Sat 06 Jun 2020 04:19:22 PM EDT 4d0320f1af9c2992a5a626100856f902  -
Sat 06 Jun 2020 04:19:27 PM EDT 4d0320f1af9c2992a5a626100856f902  -
Sat 06 Jun 2020 04:19:32 PM EDT 64f60d417fd3b165ec2cd1682198890b  -
Sat 06 Jun 2020 04:19:38 PM EDT 64f60d417fd3b165ec2cd1682198890b  -
Sat 06 Jun 2020 04:19:43 PM EDT 64f60d417fd3b165ec2cd1682198890b  -
Sat 06 Jun 2020 04:19:48 PM EDT 64f60d417fd3b165ec2cd1682198890b  -
Sat 06 Jun 2020 04:19:53 PM EDT 4d0320f1af9c2992a5a626100856f902  -
Sat 06 Jun 2020 04:19:58 PM EDT 4d0320f1af9c2992a5a626100856f902

Please log in or register to answer this question.

Welcome to MSC-Open-Data Q&A where you can ask questions about the open data of the Meteorological Service of Canada and receive answers from other members of the community.

Bienvenue sur MSC-Open-Data Q&A où vous pouvez poser des questions sur les données ouvertes du Service Météorologique du Canada et recevoir des réponses d'autres membres de la communauté.
...