<- All posts [Table of Contents]

Building a Digital Newspaper App from scratch

"newsfeed" is a side project which allows me to read daily newspaper front pages of my favorite newspaper outlets from countries around the world.

2023-09-10

The landing page shows all newspapers with a small preview. With the datepicker, you can go back in time and view older publications.

Getting your news from the internet is super easy nowadays. Simply go to your search engine of your choice and hit the tab "News" and you get the latest news for your region from all kinds of different sources.

Everything seems great until you actually want to read an article and click on one of the links and then the whole immersion collapses. The site loads, a cookie banner is popping up, different subscription models are presented, and every blank space is filled with ads.

In my opinion this is an absolutely terrible user experience! And therefore, I was looking for an alternative and discovered digital newspapers for me.

Analysis

I found a service which provides many different digital newspapers. I looked into the network traffic of the page in the hopes of finding a easy way to scrap the images.

After some research and many HTTP 403 responses, I figured out how to fetch specific newspapers based on the current date. There was only one big problem, I can only fetch the first page of the newspaper.

This was a setback, but after some thought, why should I care about news who didn't make it to the front page? If I have every day multiple newspapers to read, there is no need for additional pages.

Scrapping service

So I started building a CLI tool in Rust for scrapping the images from the service. Because every newspaper has a different image size, and changes every day slightly, I make multiple request in a specific range of sizes, preferably from larger to smaller.

Now after the scrapper was completed I set up a cronjob which executes the tool every hour from 8 AM to 11 AM. This is the time range where american newsletters on the west coast get published. When the download is complete I create the thumbnail images with the ImageMagick tool. The only thing left now was the actual webapp.

Frontend development

For the frontend I chose a combination of React and figure as framework. Because I'm not a big fan of NodeJS and the whole NPM ecosystem, I try to avoid installing the dependencies locally and instead use CDN's for the most part. Yes it is slower and more inefficient but it provides a better development experience. By using plain JavaScript instead of TypeScript the whole building and bundling steps can be omitted.

Every time I start a new project which requires React, I use my boilerplate repository to get things set up quickly.

The reader view allows focused reading without distracting UI. You can adjust the zoom level in order to read smaller texts.

Loading animation
Preview image fallback
JSON file for storing newspapers
Mobile version image

TODO

App experience

Because I want to read the news not only on my computer but also on my phone, a great addition to my existing app would be PWA support which would give the website a more app like feeling.

To transform the website into a PWA I added a manifest.json file which contains all the required information about the app. Now I am able to download my app to literally any device which can run a browser.

Because mobile devices have such a small screen size compared to desktop devices, the newspapers at this scale would be impossible to read. Therefore I implemented the ability to zoom in with the use of finger gestures. This was quiet tricky to get right with my current dialog reader view. After some tweaking and testing, I was happy with the result.

Now I can read my newspapers on the go.

Final thoughts

This project was really fun and I learned so many things along the way. I'm using the app almost daily for several months now. In all this time the cronjob, the scrapping service, everything, worked like a charm.