Simon Online

2018-09-12

Downloading a file from S3 using Node/JavaScript

There seems to be very scant examples on how to download a file from S3 using node and JavaScript. Here’s how I ended up doing it using the aws-sdk:

1
2
3
4
5
6
7
8
9
10
11
const params = {
Bucket: 'some bucket',
Key: 'somefilename.txt'
};

const s3Data = await s3.getObject(params).promise();
console.log('Content length is:' s3Data.ContentLength);
console.log('Content type is:' s3Data.ContentType);
console.log('Writing to file');
const file = fs.createWriteStream('c:/afile.txt');
file.write(s3Data.Body);
2018-09-05

Error creating an SQS queue in serverless

I ran into an error today creating an SQS queue in serverless which looked a lot like:

1
2
3
CloudFormation - CREATE_FAILED - AWS::SQS::Queue - SendEmailQueue
...
An error occurred: SendEmailQueue - API: sqs:CreateQueue Access to the resource https://sqs.us-east-1.amazonaws.com/ is denied..

You would think this error is related to something in IAM, and indeed it may well be. However in my case it wasn’t it was actually that I was giving CloudFormation a mal-formed name for the queue. I would like to think that the developers of SQS would use HTTP code 400 (bad request) to report this error but instead they use 403. So keep in mind that even though it looks like a permissions problem it might be a syntax error.

2018-07-03

Dealing with implicit any in node_modules

I like to set noImplicitAny in my TypeScript projects in the hopes it will catch just a few more bugs before they hit production. Problem we had recently was that not every library author agrees with me. In this case the aws-amplify project gave us this error.

1
2
node_modules/aws-amplify/lib/Interactions/Interactions.d.ts
(19,44): Parameter 'err' implicitly has an 'any' type.

Well that sucks, they should use noImplicitAny! From our perspective we can fix this by not applying our implicit any rules to library definition files. This can be done by adding

1
"skipLibCheck": true

To the compilerOptions in our tsconfig.json so that errors in the definition files there were ignored.

2018-07-02

Weird JavaScript - Destructuring

I’ve been at this programming game for a long time and I’ve written two books on JavaScript. Still today I ran into some code that had me scratching my head. It looked like

1
function AppliedRoute ({ component: C, props: cProps, ...rest }) {

I was converting some JavaScript to TypeScript and this line threw an linting error because of implicit any. That means that the type being passed in has no associated type information and has been assumed to be of type any. This is something we’d like to avoid. Problem was I had no idea what this thing was. It looked like an object but it was being built in the parameters?

Read More

2018-07-01

Application Insights Cloud Role Name

Logging is super important in any microservices environment or really any production environment. Being able to trace where your log messages are coming from is very helpful. Fortunately Application Insights have a field defined for just that.

Read More

2018-07-01

Application Insights Alerts

Application Insights is another entry in the vast array of log aggregators that have been springing up in the last few years. I think log aggregators are very important for any deployed production system. They give you insight into what is happening on the site and should be your first stop whenever something has gone wrong. Being able to search logs and correlate multiple log streams give you just that much more power. One feature I don’t see people using as much as they should is basing alerting off of log information. Let’s mash on that.

Read More

2017-03-25

MassTransit on RabbitMQ in ASP.NET Core

In the last post, we created an application which can send tasks to a background processor. We did it directly with RabbitMQ which was a bit of a pain. We had to do our own wiring and even our own serialization. Nobody wants to do that for any sort of sizable application. Wiring would be very painful on a large scale.

There are a couple of good options in the .NET space which can be layered on top of raw queues. NServiceBus is perhaps the most well know option. There is, of course, a cost to running NServiceBus as it is a commercial product. In my mind the cost of NServiceBus is well worth it for small and medium installations. For large installations I’d recommend building more tightly on top of cloud based transports, but that’s a topic for another blog post.

Read More

2017-03-25

Getting Started with RabbitMQ in ASP.NET

Orignally posted to the ASP.NET Monsters blog at https://aspnetmonsters.com/2017/03/2017-03-18-RabbitMQ%20from%20ASP/

In the last post we looked at how to set up RabbitMQ in a Windows container. It was quite the adventure and I’m sure it was woth the time I invested. Probably. Now we have it set up we can get to writing an application using it.

A pretty common use case when building a web application is that we want to do some background processing which takes longer than we’d like to keep a request open for. Doing so would lock up an IIS thread too, which ins’t optimal. In this example we’d like to make our user creation a background process.

Read More

2017-03-25

Some Clarity on my Thoughts on NServiceBus

In my last blog post I mentioned in passing something about NServiceBus

1
2
3
4
There is, of course, a cost to running NServiceBus as it is a commercial product. 
In my mind the cost of NServiceBus is well worth it for small and medium
installations. For large installations, I'd recommend building more tightly on
top of cloud based transports, but that's a topic for another blog post.

I though that, perhaps, there should be some clarity to my comments. I have a few good friends who make their living working with NServiceBus and there was some debate about my point.

Read More

2017-03-25

Creating a Rabbit MQ Container

Originally posted to the ASP.NET Monsters blog at https://aspnetmonsters.com/2017/03/2017-03-09-rabbitmq/

I bought a new laptop, a Dell XPS 15 and my oh my is it snazzy. The thing I was most excited about was that I’d get to play with Windows containers again. I have 3 other machines in the house but they’re either unsuitable for containers (OSX running Windows in parallels) or I’ve so toally borked them playing with early betas of containers they need to be formatted and reinstalled - possibly also thrown into the sun.

So when I found myself presented with the question “how can we get into messaging in our apps for free?” I figured I’d crack open the laptop and build something with MassTransit. I found that MassTransit supports running on RabbitMQ. Why that sounds like a perfect opportunity to deploy RabbitMQ to a container. Only problem was that I didn’t really know how to do that.

Read More