This is a single page web scraper, it uses the goquery library to parse the html and allow it to be queried easily (like jQuery). There is a Find method we can use to query for classes and ids in same way as a css selector. In our example we use this to get the latest blog titles from golangcode. If you needed to search an entire site, you could implement a query to follow and recall a link urls.
This post demonstrates how to get the IP address of incoming HTTP requests in Go. As a function, it attempts to use the X-FORWARDED-FOR http header for code behind proxies and load balancers (such as on hosts like Heroku) while falling back to the RemoteAddr if the header isn’t found. Just as an example, we created an echo server (of sorts) below to reply to incoming requests with the requesting ip address in json form.
Constants in Go, when combined with iota, can be a nice way to use options and settings as a list. We have an example below using genders to show this in action. The problem comes if you want to encode/decode structs to JSON, then the constant will be encoded into an int - sometimes it would be nice to convert this to a string to make it easier to represent what it is.
Go has a simple command line for running its tests, with go test. However, often when writing tests you don’t care about the rest of the test suite - you just want to run your new test. This post shows you the command you need to run just your test, as well as a full example below.
We have already covered pdf generation to some degree, by using wkhtmltopdf on AWS’ Lambda service. This post is about generating pdfs without needing wkhtml - by building the pdf from Go itself. To do this we use a library called gofpdf to build the pdf. It quite straightforward for simple documents, but gets more complicated the more you add to it. In our example we add some text as a title and an image just beneath it.
There are of course many different ways to build authentication into APIs these days - JSON web tokens being just one of them. JSON Web Tokens (JWT) have an inherent advantage over other methods, like Basic Authentication, by working as a token system instead of sending the username and password with every request. To learn more about it, head over to the introduction on jwt.io before we dive straight into it.
Go doesn’t have a find or in_array function as part of it’s standard library. They are easy to create however. In this example we create a Find() function to search a slice of strings for the element we are looking for. Cue the “generics” rant from some coders. This is a great example of why they could be useful. We have created our find function, but it only works with strings and you will have to create different find functions for different types - as needed.
This is a matching post to “Writing to a File” and explains how to simply get the contents of a file as text and print it to screen. There are different ways to achieve this in Go - all valid. In this guide though we’ve gone for the simple approach. Using ioutil makes this easy for us by not having to worry about closing files or using buffers. At the cost though of not having flexibility over which parts of the file we need.
Some applications and programs can be very time sensitive - and they often need to return something in a timely fashion. However, it’s not always within our control to set a cut off point these operations. Go makes this process somewhat easier though through it’s use of goroutines and channels. In the example below we execute LongRunningProcess which we’ve given 3 seconds complete - but it contains code to sleep for 5, so it will never complete.
This is part 1 of how to find the dominant colours within an image. You might spot this functionality around the internet, as it’s used to give images background colours before the image has loaded. We use a package to do this - which manages much of the heavy lifting. The package, called prominentcolor, uses the Kmeans++ algorithm to work this out. By default, it will return us the three most popular colours after both cropping and resizing the image.