Querying Azure Table Storage

Here is how to generically query Azure Table Storage without having to define the entity that is in the table (which you may not know and can vary).

CloudStorageAccount storageAccount = CloudStorageAccount.Parse(diagnostics);
CloudTableClient tableClient = storageAccount.CreateCloudTableClient();
CloudTable cloudTable = tableClient.GetTableReference(“WADLogsTable”);
TableQuery query = new TableQuery();

query.FilterString = filter;
var items = cloudTable.ExecuteQuery(query).ToList();

See my previous post for how to set the filters.

Why Value At Risk Does Not Work

The financial system is currently using the Value at Risk model.

One of the fundamental assumptions of it is that the trades used are normally distributed. The benefit of assuming normal distribution is that it is well known and easy to calculate. You can’t assume normal distribution. Is the height of a large enough population normally distributed? If you believe so where are the 6m tall humans or those less than 0m tall? These are reasonably expected to exist in a large enough normally distributed population.

Azure SDK 2.0

Azure SDK 2.0 has been released.
Here are the the release notes.

Breaking Changes:

  1. Windows Azure Diagnostics no longer requires 1.7 storage
  2. .NET Framework minimum version is now 4.0
  3. ServiceRuntime, Configuration and Caching assemblies are now built against the .NET Framework 4.0 runtime. So upgrade all apps to target .net 4.0
  4. Windows Azure Connect– The Windows Azure Connect preview is being retired June 30.
  5. Hosted Web Core Support- Windows Azure web roles now require a <Site> element in the service definition file (CSDEF) for your role.
  6. CSUpload.exe warning- CSUpload.exe now emits a warning recommending that you use Windows Azure PowerShell cmdlets to upload VHDs to Windows Azure.
  7. Service Bus client library – the Message Buffer feature, and all APIs related to MessageBuffer have been removed. You should instead use Service Bus queues.

Here are the actual code breaking changes.

Truncating Azure Logs

The following demonstates how to delete from WADLogsTable from before a given date.
It may not be the most efficient code but it does work.

This has been tested against Azure SDK 2.0

///

/// TruncateDiagnostics(storageAccount, DateTime.Now.AddHours(-1));
///

///
///
public void TruncateDiagnostics(CloudStorageAccount storageAccount, DateTime keepThreshold)
{
try
{

CloudTableClient tableClient = storageAccount.CreateCloudTableClient();

CloudTable cloudTable = tableClient.GetTableReference(“WADLogsTable”);

TableQuery query = new TableQuery();
query.FilterString = string.Format(“Timestamp lt datetime'{0:yyyy-MM-ddTHH:mm:ss}'”, keepThreshold);
var items = cloudTable.ExecuteQuery(query).ToList();

Dictionary batches = new Dictionary();
foreach (var entity in items)
{
TableOperation tableOperation = TableOperation.Delete(entity);

if (!batches.ContainsKey(entity.PartitionKey))
{
batches.Add(entity.PartitionKey, new TableBatchOperation());
}

batches[entity.PartitionKey].Add(tableOperation);
}

foreach (var batch in batches.Values)
{
cloudTable.ExecuteBatch(batch);
}

}
catch (Exception ex)
{
Trace.TraceError(string.Format(“Truncate WADLogsTable exception {0}”, ex), “Error”);
}
}

Minimal Azure Table Storage Sample

CloudStorageAccount storageAccount = CloudStorageAccount.Parse(CloudConfigurationManager.GetSetting("Microsoft.WindowsAzure.Plugins.Diagnostics.ConnectionString"));
CloudQueueClient cloudQueueClient = storageAccount.CreateCloudQueueClient();
CloudQueue cloudQueue = cloudQueueClient.GetQueueReference("test-queue");
cloudQueue.CreateIfNotExists();

Please rememeber that queue names are quite fussy. You need to use lowecase names with hypens as seperators.
It does not like brackets or underscores.
This is especially important if you use the environment name in the queue name to allow shared storage accounts across a dev team.