Introducing flickr-set-get a command line app to download photos

flickr-set-get downloads Flickr galleries from the command line using Node.js and asynchronous programming

I recently developed a small command line app that allows you to download an entire gallery from Flickr, it’s called flickr-set-get and you can find it on NPM and GitHub.

a screenshoot of flickr-set-get

Why?

To be honest I had myself the need to download a large set of photos (more than 400 photos) from Flickr and I didn’t wanted to do it manually. I also wasn’t able, after a quick search, to find something simple to solve this task. Given that I am currently getting into deep of Node.js this was the perfect chance to develop something practical.

How it’s built?

As I said it’s built using Node.js. As part of my learning process, I tried to not reinvent the wheel and to follow “the Node way” so I used some very good modules as the foundation to build this app:

  • Commander, Cli and Prompt for the command line facilities (Options/Arguments parsing, help generation, progress bar, receiving input from the user)
  • Request: to simplify the creation of the Http request to call the Flickr APIs
  • Async: to manage all the asynchronous tasks in parallel and with a configurable level of concurrency (one of the features that in my humble opinion makes node/javascript stand out from the crowd of the interpreted programming languages).

As development tools I also used:

  • JSCS: to check the code standard I adopted
  • Mocha and Chai: test runner and assertion frameworks for the unit testing
  • Nock: amazing module to “mock” the Http requests and avoid to hit Flickr servers in my tests
  • Istanbul: for the code coverage
  • Travis CI: continous integration service
  • Coveralls: service to keep track of the coverage changes after every new commit
  • Gemnasium: service that checks your dependencies and alerts you if they get out of date

Going asynchronous and execute requests in parallel

To download a photo from Flickr, starting from a gallery, you need to make several API calls before you have the right URL to download the photo.

First of all you need to call the flickr.photosets.getPhotos API method to find out the IDs of all the photos in the set. Then, to find out the URL of every photo you need to call the flickr.photos.getSizes API method. This method gives you all the URLs to download all the different sizes of a given photo. Once you got the URL you can simply use that to download a photo into your local drive. The first API call is the starting point and then, since you now have all the IDs, you can make all the other requests in parallel. Thanks to Async it was very easy to create a task function that figured out the specific URL of a given photo and then downloads it into a file. Then I just needed to replicate this task for every photo and run all of them in parallel. Thanks to the Async.parallelLimit() function it’s even possible to run the tasks with a configurable concurrency level.

flickr-set-get asynchronous requests diagram

You can find out how I implemented this specific logic on GitHub: https://github.com/lmammino/flickr-set-get/blob/ecaf0d8bea5791cf24c4b71791a7eb1fd2e60bcb/lib/Flickr.js#L212-L253.

How to use it?

If you’re interested on using it I suggest you to read the official documentation on GitHub, but it should be quite easy to understand and use it if you are a developer (and it’s not meant to be used by other people who are comfortable with the command line!)

Current status

The project has been developed in a weekend as a “learning project” and tested just by me on few Flickr galleries. It’s to be considered ALPHA quality and I expect it to be fully of bugs and uncovered edge cases. It’s a small side project anyway so I don’t expect to put so much effort on it again unless it gets adopted by a relevant number of people (which, quite frankly, is really unlikely to happen). Being an open source project I obviously tried to put in place all the basic infrastructure (tests, code standards, repository, versioning, etc.) to make it easy for other people to work on improving it. Again, more precise informations are included in the official documentation.

Opinions

For me it has been a really good learning experience, especially considering the fact that it was a good scenario to use asynchronous code dealing with several parallel Http requests. As I still consider it a learning experience I would really love you (probably more experienced than me with Node.js) to give some opinions, critics, new ideas. I offer the comment box below for this sake… please don’t be too much indulgent :D

Sharing is caring!

If you got value from this article, please consider sharing it with your friends and colleagues.

Found a typo or something that can be improved?

In the spirit of Open Source, you can contribute to this article by submitting a PR on GitHub.

You might also like

Cover picture for a blog post titled Migrating from Gatsby to Astro

Migrating from Gatsby to Astro

This article discuss the reason why I wanted to migrate this blog from Gatsby to Astro and the process I followed to do it. Plus a bunch of interesting and quirky bugs that I had to troubleshoot and fix along the way.

Calendar Icon

Cover picture for a blog post titled Middy 1.0.0 is here

Middy 1.0.0 is here

The middleware framework Middy reached version 1.0, bringing middleware capabilities to AWS Lambda. This allows cleaner handler code by extracting cross-cutting concerns into reusable middleware.

Calendar Icon

Cover picture for a blog post titled Lean NPM packages

Lean NPM packages

Learn how to configure NPM packages to publish only the files needed by users, avoiding bloating node_modules folders.

Calendar Icon

Cover picture for a blog post titled Fastify and Preact for quick web app prototyping

Fastify and Preact for quick web app prototyping

This article shows how to quickly build web app prototypes using Fastify for the backend API and Preact for the frontend UI. It also covers how to dockerize the app for easy sharing. Key points are the plugin architecture of Fastify, the lightweight nature of Preact, and the use of htm for defining UI without transpilation.

Calendar Icon

Cover picture for a blog post titled JavaScript iterator patterns

JavaScript iterator patterns

This article explores different ways to create iterators and iterable values in Javascript for dynamic sequence generation, specifically using functions, iterators, iterables and generators. It provides code examples for implementing the Fibonacci sequence with each approach.

Calendar Icon

Cover picture for a blog post titled Emerging JavaScript pattern: multiple return values

Emerging JavaScript pattern: multiple return values

This article explores how to simulate multiple return values in JavaScript using arrays and objects. It covers use cases like React Hooks and async/await error handling. The pattern enables elegant APIs but has performance implications.

Calendar Icon