@misprintedtype

I’M OFFLINECOOL, WHAT NOW?

- we <3 GOTOBerlin -

Hello there!

How are you? Are you tired or still good? Did you enjoy your first day so far?

Cool! Okay…

So, lets do this. Welcome to my talk about best practice in offline first and the learnings we gathered in the past few years.

OLA GASIDLO

Lead Developer

Co-Organizer

CSSConfEU

My name is Ola Gasidlo. I am a lead developer at Styla, Co-Organizer of CSSConfEU and organizer of various conferences and meetups and even more…

Today I’d love to tell you a bit about the actual state of offline first, how our point of view changed in the past few years and how its adapting to the real world.

@misprintedtype

Trigger Warnings

death / pandemic

animated gifs

Before I start, I’d like to mention in this talk I’ll speak a pandemic and death, but there won’t be any kind of graphical display of those topics. Plus there will be

animated gifs.

@misprintedtype

Trigger Warnings

death / pandemic

animated gifs

So, if you’d rather leave, because you won’t feel comfortable with this content… I fully understand.

I am happy to chat with you after the talk and give you a summary of the content. You still have time to leave, if you’d like to until the first gifs. Let’s start then.

@misprintedtype

The internet, it turned 25 this year. Amazing times we all live in…

@misprintedtype

In this time, our relationship with the internet changed. A lot.

We do different things, with different devices than 25, 10, 5 years ago... and it is a part of our everyday life! We rely on our devices... the internet, but our needs changed.

@misprintedtype

Three years ago, when the Offline First movement got broader attention, I gave my first talks about the concept. That was in twenty-thirteen.

What was our idea back then…? Why did we need the offline first concept so badly?…

@misprintedtype

First thing was the rise of the mobile devices and their usage. There are countries, where people do not even use desktop devices at all anymore.

When you think about how cheap a phone is in comparison to e.g. a laptop, it totally makes sense, right? A mobile device is mostly enough, if you consider what the

user needs it for.

@misprintedtype

There was also the issue of no constant connectivity or poor network coverage.

Here offline first felt like a huge enhancement to your application and adaption to our new behavior.

@misprintedtype

Three years later, in twenty-sixteen, what do you think..?

Do we still need the offline first concept?

@misprintedtype

TELL ME…

Tell me… (by raising your hand) - Who of you are already building offline first applications? - Who of you thinks, they are following the correct approach to make their applications Offline First? (There is still no perfect path… We are still trying to figure this one out.) - Who of you thinks about Offline First as progressive enhancement?

@misprintedtype

After seeing offline first in the wild, we’ve realized…

it’s not just about a special use case or rise of the mobile devices.

It’s about the whole approach… It’s so much more.

„OFFLINE FIRST“ MEANS GETTING ON SCREEN WITHOUT A NETWORK REQUEST…

Jake Archibald @twitter

As Jake Archibald once said…

„Offline first“ means getting on screen without a network request…

…THE MORE THAT’S THERE BEFORE THE NETWORK, THE MORE OFFLINE FIRST IT IS.

Jake Archibald @twitter

…The more that’s there before the network, the more offline first it is.

This means, every web app should work without a connection after the first visit and just use the network for getting / setting new data. Everything else is online-first.

@misprintedtype

PWA?Remember me asking you if offline first is a progressive enhancement to your application? Like 3 slides ago? What if I tell you, you are actually…

@misprintedtype

WRONG…wrong! -> 1, 2, 3

This might sound confusing… Why is the offline first concept not progressive enhancement? Actually it is the other way around…

Let’s take a step back and see, why do we need applications?

@misprintedtype

WHY?Normally we use applications for our everyday life, to make our everyday life easier, right?

Like…

@misprintedtype

Checking in with family members and other loved ones abroad or maybe just text your partner during the day.

@misprintedtype

Using Slack, IRC etc. at work, especially in remote teams.

-> it all comes down to…

@misprintedtype

COMMUNICATION

-> communication

@misprintedtype

We also need applications for tracking data.

Did I drink enough during today?

How many steps did I make?

How much money did I spend this week?

But also…

@misprintedtype

How can I fix my bike or my washing machine?

How do I translate this or even… „Did I spell this right?“

-> basically it’s about

@misprintedtype

COMMUNICATION INFORMATION

-> information

@misprintedtype

Posting selfies from your last vacation, tweeting your thoughts etc.

Checking in symptoms when you are sick…

@misprintedtype

Or even check the internet why your kids act out.

-> this is about

@misprintedtype

COMMUNICATION INFORMATION EMPOWERMENT

-> empowerment

AND… most important thing…

@misprintedtype

cat.gifs… (1… 2… 3…)

-> Which is a basic life need. I think, we all can agree on that. We sum this one up as

@misprintedtype

COMMUNICATION INFORMATION EMPOWERMENT LOVE

—> love.

So, when we look at those points…

Communication, information, empowerment and love, this all sounds like something we all have in common right?

@misprintedtype

When we break it down, it all comes down to help satisfy our very basic, humane needs for safety, belonging, esteem and self-actualization.

Maslow defined all those points in his „Hierarchy of needs“ already in nineteen-fourtythree.

@misprintedtype

COMMUNICATION INFORMATION EMPOWERMENT LOVE

So since when is helping people to fulfill basic needs, progressive enhancement?

@misprintedtype

// drauf zeigen… sounds familiar right?

But we love to sell it that way, so developers jump on the progressive enhancement train and just redo their apps every 6 weeks in the new, cool framework or

whatever. Without knowing what the real issue is…

@misprintedtype

// drauf zeigen… nech?

To be honest… I am so tired of half done applications, which are released out in the wild without any kind of documentation, because someone wanted to refactor their app in the „new cool stuff"…

@misprintedtype

// drauf zeigen… -> show quotation marks…

Or super slow apps, b/c of bad planing and messy architecture.

@misprintedtype

RETHINK! We do not need a new UBER or Twitter… REALLY. Trust me!

We tend to forget that THIS, our JOB is all about humans, people… not frameworks, tech or a design pattern, or the discussion about semicolons.

@misprintedtype

Don’t build a town, when you need a house. Just because you can!Especially because usually you end up with the „Gorilla Banana Problem“.

„YOU WANTED A BANANA BUT WHAT YOU GOT WAS A GORILLA HOLDING THE BANANA AND THE ENTIRE JUNGLE.“

Joe Armstrong

-> read quote

The Gorilla / Banana problem: Every time you use a whole infrastructure like a framework to solve one problem, you get all this implicit environment that they carry around with them.

It’s about using the right tool for a job. Without building a town, when you just need a house. Just because you can.

@misprintedtype

COMMUNICATION INFORMATION EMPOWERMENT LOVE

It’s all about keeping things simple, but solid. Basic, but optimized.

This is why I think, putting offline first in the progressive enhancement slot is pure BS.

@misprintedtype

Do you know, what the progressive enhancement part of your application is? The network.

You might ask yourself… Why should I care? My apps are working…

@misprintedtype

Are they? (1… 2… 3…)

Ever checked your application without or with a bad connection?

@misprintedtype

How productive are you when e.g. just one service like GitHub is down? Not to mention the office WiFi? How easy is it to communicate with your friends at a concert,

when everyone is sending a live stream of the concert via their phone?

WEALTHY WESTERN WEB

Bruce Lawson

And besides us - with good connectivity situation blessed people, there are others…

@misprintedtypehttps://speakerdeck.com/benschwarz/vanity-performance-metrics-looking-beyond-the-numbers

Just 43% of the population has access to an internet connection. And just a small amount of them to a fast and steady one.

@misprintedtypehttps://speakerdeck.com/benschwarz/vanity-performance-metrics-looking-beyond-the-numbers

And data is expensive… As you can see here on this list from Ben Schwarz’s talk about performance metrics. Germans would have to work less than one hour for 500

MB, but people in India about 17 hours…

500MB PER MONTH

Bastian Albers

Let’s assume you have a 500MB plan…

2,4MB AVERAGE PAGE SIZE = 6,94 SITES/DAY

Bastian Albers

So you could visit almost 7 sites per day during that month.

~22MIN 720P VIDEO

Bastian Albers

Or watch 22 minutes of talks… So basically ONE talk per month and your data would be gone.

TENS OF THOUSANDS OF EMAILS

Bastian Albers

https://www.whistleout.com.au/MobilePhones/Guides/Mobile-broadband-usage-guide

Or you could make your others really happy and finally write ALL the mails that you are procrastinating… Without attachments of course.

@misprintedtype

Our applications should provide the base and work offline per default. The network is the one to enhance our application with new, outside data.

@misprintedtype

No matter which technology stack we use… Peer-to-Peer network, direct connection to the server or a USB-Stick, whatever works to transfer the data.

It’s not important HOW you transfer the data or application on a device. What is important is, we have saved it all locally and enhance the sync through the network.

@misprintedtype

the REAL world

Not sure, this is true? Let’s check out what is happening in the real world and how others solve their offline issues.

@misprintedtype

the Router

Let’s start with… the Router…

@misprintedtypehttps://ingevald.files.wordpress.com/2009/02/sweden-river-fog-morning-looking-at-sun.jpg

Imagine… A beautiful morning.

You’ve decided to work remotely and are sitting in an „internet cafe“. You’d love to check your mail? Cool!

Connect your laptop to the router. Open your email app and send the request to get your mails. Close the laptop and come back in the evening… wait, WHAT!? Why

you might ask?

@misprintedtypehttps://ingevald.files.wordpress.com/2009/02/sweden-river-fog-morning-looking-at-sun.jpg

Well… There is no „physical“ connection to the internet like a cable that the internet cafe is using. They just have a router that saves your request and connects via

bluetooth to a…

@misprintedtype

bus!

It syncs the requests and off it goes. Literally.

The bus arrives at a stop after half of the route to the hot spot with physical connection and syncs with another bus. Also exchanges not just the data, but also the

passengers and luggage.

@misprintedtype

The buses sync via bluetooth previous requests and results. Both buses return. Literally and yeah… you get your emails.

Hopefully. If you were able to authorize.

@misprintedtype

(SNC - Service)

Using NAT zones, DTN routing

&& DakNet network.

Saami Network Connectivity

The tech behind the SNC Service is a combination of NAT zones, DTN routing and the DakNet network. (Not to be confused with the DarkNet.)

@misprintedtype

Learnings (requests & assets)

So, what are our learnings here?

@misprintedtype

Check your requests!

Actually check your requests, because beside you should be aware of what your are actually doing…

Do you need 20 requests or would actually just 2 be enough? Please serve uglify and minified files. Do you really need that 5MB background image on mobile?

@misprintedtype

beware of Lie-Fi

You should be aware of Lie-Fi, which is the state, where your device checks your connectivity situation, assures that you are online (pinky promise), but there is actually

no data passed through.

@misprintedtype

data queries with bandwidth?

We could go with checking the bandwidth after the very fist ping right? Yes, that could be a thing!

And we could develop something similar to media queries and device sizes but just with bandwidth. So we can load assets per detected TRUE bandwidth.

@misprintedtype

Great idea, right?!

But the bandwidth can differ per request, even per package, especially when you are moving, so e.g. are on a train.

@misprintedtype

data queries with bandwidth

We would need to ping not just per every request, but constantly. You can’t rely on requests to come back.

Please add a timeout to kill your request or make the app work fully independent of requesting data.

@misprintedtype

enhance assets with network

This is why we need to separate updating cycles of the assets and the data. The update of assets is less important than actually syncing data.

@misprintedtype

save basic set of UI locally

Save a basic set of your UI assets locally (to go easy on your storage).

Enhance it with e.g. web fonts, images etc. Whatever you need… Offline should be the default state and we need to make the bandwidth assumption by the device

we detect.

@misprintedtype

keep the caches clean

Take responsibility for the caches and keep them clean.

What could be better to use to manage your caches, than Service Worker. No intro in Service Worker today. Sorry. Just one thing…

@misprintedtype

If you want to use the AppCache for quite a bit, because browsers are dropping the support hard, but also for migration from AppCache to ServiceWorker, please

check out the JakeCache.

@misprintedtype

SVG over IMG

Images… Hard to deal with within the offline context…

Saving images in the in-browser storage? Good way to go, but it is still very painful because of storage limitations and very different storage limit implementations.

Porting decorative images, like icons and logos to SVG and CSS is a great way to go to cut off a bit of the overhead.

@misprintedtype

base64 over IMG

Encoding assets in base64 and storing the String in the in-browser storage of your choice could be a way to go.

For example, saving fonts like that and loading them straight from the storage, improves the performance a lot. This is even possible for images, like a logo. But

please, just do not do this for all of your images. Don’t build the town, build the house. Remember? The strings are huge as we all already know. And in-browser storages are is limited.

@misprintedtype

base64 is expensive && heavy

Also decoding base64 on the fly is a heavy task.

So it slows down your site. Especially in combination with DOM-manipulating frameworks like React. You’ll also find yourself running into race conditions here.

-> drink

Let’s check out the next use case… It is a bit heavier, as already mentioned. Just to let you know in advance.

@misprintedtype

the Pandemic

-> the Pandemic

@misprintedtype

Twenty-fourteen and twenty-fifteen, the ebola pandemic in parts of West Africa was a really scary issue.

@misprintedtype

I was involved in a project by the NGO eHealth Africa that build two applications to help stop the pandemic.

The first application to solve one of the issues was… providing information for the population, because information is the key to prevent a pandemic to spread or kill.

@misprintedtype

The process was… The population was informed about the number they could call to ask about ebola symptoms, to report someone sick or even dead.

The call center tracked the calls with the application we’ve build, collected all the provided information from the caller and sent out a team, if needed.

@misprintedtype

They added information through the application like the name, location and additional information. Also added the phone number of the caller, so they could call back for any further questions.

Data analysis and filtering was really easy thanks to the application.

@misprintedtype

rm -rf ~/

They fought with issues like unexpected and constant power loss. The generators jumped in after some time, but we could not afford to lose data. Just one case

could have caused another outbreak and many deaths.

@misprintedtype

first learning (data)

Our first learning here was…

@misprintedtype

Decouple server && client

Decouple server and client, to serve the needs of a eventual contingency.

@misprintedtype

build a task / message system

Build in a task and message system, that can act independently from the connectivity situation.

For example… for data CouchDB, especially in combination with PouchDB proved their usability in the real world really, really well.

@misprintedtype

CouchDB a NoSQL database that replicates, which is exactly what we want for sync.

CouchDB comes with a suite of features, such as on-the-fly document transformation and real-time change notifications, we need for our message and task system,

but also for a decoupled architecture.

@misprintedtype

PouchDB is an open-source JavaScript database inspired by CouchDB that is designed to run well within the browser and store all your data locally while offline.

@misprintedtype

It sync’s with CouchDB thanks to the CouchDB replication protocol, when the application is back online.

@misprintedtype

save data documents locally

With PouchDB we actually have documents which are stored locally. So we won’t not flush the caches during by accident.

@misprintedtype

prevent merge conflicts

One of the biggest issues with syncing data are merge conflicts.

Because PouchDB and CouchDB speak the same protocol and sync automatically when we are back online and we do not need to take care of it.

@misprintedtype

The second application we’ve build, was to actually track patient data in the field. After examining the infected patients, who had to stay in quarantine for 21 days.

The team added the data to a general database through the application plus got lab results from several different entry points and very different file types. 

@misprintedtype

The data was imported, saved in a global database and synced all across all kind of devices. This all sounds super messy, right? What do you do with a mess?

Well, I am a parent. So I am used to cleaning up a mess…

@misprintedtype

second learning (schema)

Our second learning here was…

@misprintedtype

define flexible data schema

We added an organization system. An interface to map the data in one defined schema. The schema was flexible enough to be extended, if needed… Thanks to the

NoSQL database behind it.

@misprintedtype

test data schema constantly

Data was imported automatically when it arrived (mostly per mail) or we added data on the fly. Constant automated testing and a high test coverage for the importer and the interface was a must.

@misprintedtype

prevent merge conflicts

The schema and the testing helped us a lot here to prevent merge conflicts before the get go.

CouchDB and PouchDB helped us to take care of the sync, merge and flexibility.

@misprintedtype

prevent merge conflicts

Big bonus also? The schema also helped us with making filter and analyze the data, so visualizing it was really easy.

@misprintedtype

10/2014 12/2015

We helped to curtail the pandemic, because of clean data structure and constant testing of the applications.

If anyone tells you that those things don’t matter or you don’t have any time for those, you now know what to reply.

Almost lunch… Guess, well picked theme for the next use case. Sorry in advance…

@misprintedtype

the Pizza

-> the Pizza

@misprintedtype

We all love to order food, right?

Did you ever check the localStorage when ordering? Just to see, if they save the cart items in it?

A friend of mine did this once and went bit further out of curiosity. He was wondering if they are actually validating their checkout on the server.

@misprintedtype

So he manipulated the checkout and subtracted one cent, send it out and waited.

The pizza came. And no, there was no server side validation.

@misprintedtype

Learnings (security)

What do we learn here?

@misprintedtype

validate on server

In contrast to the limited possibilities of our use case before, this company could have validated the data on the server.

@misprintedtype

validate on server

But this is sometimes not possible, especially when everything needs to work without any requests.

Security is still a very new topic to offline first and not many tools are provided yet. And I’d be happy to chat especially about that topic after my talk.

@misprintedtype

CORS <3

Although many of us seem to hate CORS sometimes, it provides already a great set of rules to make web applications more safe. But it does not solve the issues with

the in-browser storage manipulation.

@misprintedtype

encrypt local data

One way to go would be to encrypt your data. When you use PouchDB, you can add CryptoPouch, which encrypts your Pouch documents.

Big downside here is, the document get’s encrypted, but not the ID. Which can lead back to security issues. But at least you can’t change the data anymore in the

console.

@misprintedtype

When you want to encrypt the data, you might want to log in. Which does not work without a request to the server, right?

There is a way to use tokens in the client.

@misprintedtype

JSON web tokens are an open industry standard method for representing claims securely between two parties, which is exactly what we want!

It makes decoupled and secure sync and communication possible.

@misprintedtype

BONUS 🤗 🚀 🎉

BONUS LEVEL!!! Because this is pretty much the most creative use case I’ve ever heard to solve the offline issue.

@misprintedtype

the Rally

The Rally.

@misprintedtypehttps://en.wikipedia.org/wiki/Mongol_Rally#After_the_rally

Anyone of you heard about the Mongol Rally before?

It’s an unconventional rally where people start from London or the Czech Republic and drive down to Mongolia. The main intention of this rally was a „donate your

car“ program where every car that makes it to Mongolia, can be donated without import duties.

@misprintedtype http://www.minicong.com/lo-tec/

2009 a team from Spain took part in the rally.

Nothing special at first glance… Well.

The issue at the rally was, they had to cross a LOT of countries. With different receptions, different payment models for data volumes etc. So they needed a way to

communicate with their loved ones, so they could say… „We are alive and well.“

@misprintedtype

They built a system, that could encode an ASCII message in 8 bit binary code, ran the bits through 8 phones, that actually called just one number.

@misprintedtype

1

2

3

4

5

6

7

8

1 1

1 1

1 1

1 1

1 1

0 0

0 0

0 0

To save money, they didn’t establish a full connection. It was mainly about letting it ring. The short ringing was a zero, the long ringing was a one.

The other end, checked the numbers, which were set for order of bits and decoded the binary back in ASCII. Then it sends out message to loved ones. <3

Mind blowing, right? Yeah, people… :D

@misprintedtype

Learnings

What do we learn here?

@misprintedtype

Rethink how to use technology,

by thinking about the issue.

Not about the tools.

Read quote…

Unidirectional communication. Very basic.

They fixed it all already in the beginning by thinking about the real issue and not the solution first.

@misprintedtype

You rock!

Okay, folks… Looks like our time is up. I hope, you had a great time and thank you for listening. If you have any questions, let’s talk.

Otherwise please reach out via twitter or find me here somewhere.

And remember…

@misprintedtype

It’s not about frameworks… or design patterns or tools…

@misprintedtype

… it’s about humans!

@misprintedtype

we <3 you!

Thank you.

@misprintedtype

And please remember to rate this session OR give feedback in person.

Thank you!