Delivering ARPA Piemonte meteo-alert bulletin using Azure services

ARPA Piemonte is the regional environment protection public agency of Piedmont region, north-west of Italy. Among its duties, everyday it publishes a meteorological alerting bulletin mainly focused on meteorological / hydrogeological risks. The bulletin is the base data upon which Civil Protection agency manages risky situations like heavy thunderstorms, flooding, landslides, etc. The regional territory is divided into 11 areas in order to have more detailed and relevant information. The bulletin is available as pdf - for human accessibility - and as XML file - for system integration - following the CAP-XML standard. In addition, ARPA Piemonte also publishes a “Meteo Vigilanza” bulletin that highlights main meteorological phenomena of the following days: heavy rains, strong winds, snow, etc. Even this bulletin is available in a structured format, as a json file, downloadable from the web.

Since information available in the two bulletins can be very useful for citizens, PMA - Piemonte Meteo Allerta application has been developed. The aim of the application is to regularly retrieve data from ARPA Piemonte, transform them into human-readable bulletins and deliver them as emails or web-push-notification to subscribers. Azure cloud services has been widely used to store data and host the application. The front-end web application is available at https://meteo.fhtino.it/

Architecture and building blocks

The application must regularly check for new data from ARPA Piemonte, download it, and deliver it by email and/or web-push-notification to subscribers. Every subscriber receives the bulletin for the zone they opted for. Optionally, the subscriber can choose to receive the bulletin every day or only in case of an actual alert (“yellow” alert level or above).

The application is composed of the following elements:

  • .NET 6 Azure Functions app in charge of retrieving data, processing it, and submitting emails, web-push-notifications and Telegram messages.
  • .NET 6 web-application for viewing current data and subscribing to the service.
  • Cosmos DB container for data storage.
  • An Application Insights account for performance monitoring.

External services involved:

  • SendGrid for email delivery
  • Web-push-notification services from browser producers
  • Telegram Bot API
  • WKHtmlToImage for converting html to png image

Processing

The whole activity is split into simpler and atomic activities such as getting the data from ARPA, processing xml data, delivering emails and notifications, etc. To do that, the app running on Functions implements a command pattern on a queue. Every command is responsible for simple and fast operations. If long executions are required, the app has been designed to split them into many commands. E.g., sending 100 emails is divided into 100 single-email-delivery commands. Every command is stored on an Azure Queue as a json message:

{"Name":"CMD_SEND_EMAIL","Parameters":["13_2019","Piem-L","user@domain"]}

The Azure Functions application executes 2 functions:

  • CheckARPA: regularly enqueue a “check_arpa” command in the processing queue
  • ExecCMD: executes commands stored in an Azure Queue.

When a message, containing a command, is available, the queue triggers the ExecCmd function that receives the command as input, executes it and returns an optional list of output commands. The output commands are then inserted into the same queue, triggering new calls to ExecCmd.

This approach allows Functions to distribute the load to different instances and scale horizontally.

Commands handle:

  • Getting data from ARPA.
  • Composing html ready bulletins.
  • Converting html to image.
  • Fan-out delivery process.
  • Processing the subscribers.
  • Delivering emails.
  • Delivering web-push-notifications.
  • Send Telegram message

Subscription

A public web site allows users to freely subscribe to Telegram notifications or to web-push-notifications. They can subscribe to the area they are interested in.

A “BOT” manages Telegram subscription and delivery. User just sends a message to wake up the bot and the BOT drives them to subscribe. Sending a message with code of the area of interest (“a”, “b”, etc.) is enough to subscribe or unsubscribe.

The other subscription option is Web Push Notifications. Notifications are delivered through web-browsers, but support is very different among them. Some do not implement notifications at all, notably Safari and Chrome on iPhone and Mac. Others, like Microsoft Edge, Google Chrome and Firefox, have similar overall implementations but show different behaviors receiving and showing notifications. As far as I can see, Chrome and Edge can show notifications even with the browser not running, only on recent version of Windows 10. On the opposite, Firefox requires browser already running. On Android, Chrome works as expected. At the moment, there is no solution for iPhone. In general, “installing the app” when suggested by the browser, does not change the overall behavior. Notifications appearance is slightly different based on the system the user used for registering: personal computer or smart-phone.

The bulletin can also be delivered by email. Currently that service subscription is restricted to a small test group of friends.