Send Data into Azure Event Hubs using Web Api's HttpClient
We've recently been working on an Internet of Things (IoT) Proof of Concept which involved sending device telemetry into Azure to be shaped, enriched, materialized into different data formats and then computationally processed, before being ingested into PowerBI.
As another part of the Proof of Concept was to design and manufacture the actual device, we knew we'd need to decouple these work streams and build a device emulator. As part of the data we wanted to collect was based on GPS and accelerometer data, we decided to create a simple Windows Phone application to use the inbuilt sensors to collect this type of data and then mock out the other data series we were interested in collecting.
Unfortunately there currently isn't a Azure Service Bus client library that will run on Windows Phone and as the physical device we were prototyping also needed to be able to communicate to Azure Event Hub we needed a way to send the data via standard HTTP protocol. Luckily Azure Event Hubs (which is just Service Bus under the covers) exposes a ReST API and the platform layers abstractions like Brokered Messages on top of HTTP (and AMQP) to provide a simpler API.
The minimum requirement for sending a message to Azure Event Hubs involves sending a serialized payload to the correct queue path, which is of the format
https://NAMESPACE.servicebus.windows.net/EVENTHUB-NAME/publishers/PUBLISHER-NAME/messages
and that the HTTP message must have an Authorization header which contains a SharedAccessSignature token based the Shared Access Policies you have configured for your event hub, in this example we have two – one for the device sending data into Event Hubs (the producer) and another for the processor (the consumer):
The next step is to make use of Sandrino Di Mattia's very helpful Event Hubs Signature Generator tool, which will generate your signature based on the following information:
- ServiceBus Namespace
- Event Hub name
- Publisher name
- Sender Key Name ("device" in our screenshot above)
- Sender Key (the primary key from the shared access key generator section for the policy for "device" – see screenshot below)
- Mode (HTTP or AMQP) – choose HTTP
- TTL – because this key will sit on the device, you either need to set a very long TTL or implement a mechanism to update this SAS token
Once you have this Shared Access Signature, you can set the request header, serialize the payload and use the Web Api HttpClient to make the request:
private Task<HttpResponseMessage> PostTelemetryAsync(DeviceTelemetry deviceTelemetry)
{
// Use Event Hubs Signature Generator 0.2.0.1 to generate the token
// https://github.com/sandrinodimattia/RedDog/releases/tag/0.2.0.1
// http://fabriccontroller.net/blog/posts/iot-with-azure-service-bus-event-hubs-authenticating-and-sending-from-any-type-of-device-net-and-js-samples/
var sas = "SharedAccessSignature sr=YOUR TOKEN HERE";
// Namespace info.
var serviceNamespace = "YOUR NAMESPACE";
var hubName = "YOUR HUB NAME";
var url = string.Format("{0}/publishers/{1}/messages", hubName, deviceTelemetry.DeviceId);
// Create client.
var httpClient = new HttpClient
{
BaseAddress = new Uri(string.Format("https://{0}.servicebus.windows.net/", serviceNamespace))
};
var payload = JsonConvert.SerializeObject(deviceTelemetry);
httpClient.DefaultRequestHeaders.TryAddWithoutValidation("Authorization", sas);
var content = new StringContent(payload, Encoding.UTF8, "application/json");
content.Headers.Add("ContentType", DeviceTelemetry.ContentType);
return httpClient.PostAsync(url, content);
}
If you want to test that messages are successfully being written into Event Hubs, Dan Rosanova has a very nice two part series (1, 2) on Event Processor Host best practices. Event Processor Host is a simple framework for processing high volumes of Event Hub data.
Sign up to Azure Weekly to receive Azure related news and articles direct to your inbox or follow on Twitter: @azureweekly
