Implementing sockets with NodeJS and the journey from novice to proficient

By Samuel Muiruri | Dec. 20, 2018 | NodeJS


Not to long ago if you told me to learn NodeJS you’d see me raise an eyebrow, mainly because for me Django is an all you can eat buffet, why should I move?

Then I tried implementing sockets on Django, note that Django is built to work with a http request/response cycle so implementing it on the front end is easy, the backend though, is another story.

The best Django app that can help you implement sockets is django-channels, I’ve not yet been able to write a project and add sockets and get it to work. You can have a look at the channels documentation and compare it to socket.ioused in NodeJS, there’s also an implementation of socket-io in django, according to its README it’s not really up to date but might work.

Now when I eventually started to learn ReactJS, VueJS “still implemented these in Django” and some other Udacity courses I’d find most repo’s use Node as the backend server, when I finally got around to learn it I was amazed to learn what Node really is: it’s javascript running on top of C on you’re PC. So thanks to some C bindings it can now read and write files, spawn processes… It’s also not some new-age javascript like TypeScript.

It’s also got it’s own repo of tools like pip in python called npm and when you install something it downloads and installs it the node_modules folder. Same in python you have the main lib folder for installed scripts and you’re own virtualenv so you can isolate what only the package needs. You install a dependency using npm install <package_name> and you add the --save to add it to package.json as a dependency and --save-dev to add it as a developer dependency.

With node you can create a project by creating an empty folder then run from the terminal in the same directory run npm init and it will create a package.json that will ask you for the app name, author, and so on and that’s a Node app. You can handle post and get requests without having to write some custom code or a lot of hassle using express, install express using npm install express --save. You create a file Node’s expects to be the main to run, the default is index.js or you could use app.js . You can also install express generator npm install express-generator -g the -g flag installs it globally, which comes with a command-line tool. You can then have express create some an app for you much like django’s startproject with a default page with Welcome to Express , static folder created and configured, ready to serve static files using express --view=pug <app_name>.

If you use the generator this command note --view=pug installs another dependency pug which is like jinja on Django as a template framework that dictates how html templates are written and extends from another template. pug has one quirk though, you don’t write html normally, you instantiate the element like div, p, span with a dot notation for all classes linked to it and in brackets it’s other attribute like href, src, id… and then at the end it’s content. It’s also space sensitive like Python so you indent blocks to mean anything inside it’s block should be nested inside. So a normal pug file located in viewsfolder looks like this

extends layout
block content
  div.container
    div.row
      div.col-md-4.login-box
        label Please Provide Username
        input.username.margin-right(type='text')
        button.btn.btn-primary.login-btn Login
        p.hidden.login-error.text-danger Please type your username into the field
    div.row.chat-box.hidden
      div.col-md-2.sidebar
        ul.user-list
      div.col-md-9.content-box
        div
          span.badge.badge-secondary.typing
          textarea.form-control.message-box

There’s some other available options and express can use a template on it’s own but it might lack the feature of extending from another template, I’ve honestly used pug for now & expect I’ll write another article on this later on. Also using npm publish you can publish you’re app to the npm repos so anyone else can install it using npm install <app_name> making it trivial to publish an app, easier than python to pypi “just my opinion” and likely how it got so many apps even rival to some extent Django & Python packages. package.json also comes with version starting at 0.0.0 so if you also make any changes to a repo you’re maintaining on npm you should consider bumping the version numbers. The rule of thumb being minor changes fall under a main version so you bump up the small digits and for major changes which likely would cause backward compatibility issues you bump up the large digits.

Node normally won’t restart if you make changes like django “running locally”so I also used nodemon which keeps track of file changes and will restart Node whenever any .js file is edited and saved. To run the project using nodemon you use nodemon <app_name> also installing it with --save-dev so it saves as a developer dependency.

Up to here, I hope I’ve explained well how it’s like transitioning to using NodeJS having used other languages and knowing some javascript.

Current Final Version

My intent was to get sockets working so apart from express and it’s dependencies I also installed socket.io and the above is the current final version. Since socket.io is fairly simple on the front-end you include the scriptabove the script you’ll make a connection, I’ve also got some other dependencies namely jQuery, jquery-cookieand bootstrap. The expressapp now needs to connect with socket, the auto created express app however has it’s run server command in bin\www not that it’s a bad idea since I assume it’s like php where it also has could also have a front-end handler like apache or nginx to handle static files. Binding to it wasn’t working with sockets so I created an additional “server” on port 4000 on top of it’s default 3000, the site though loads and works on both ports. This is still from a newcomer perspective so probably a seasoned NodeJS dev could fix this.

To understand this here’s a few lines from the app.js file

//imports express
var express = require('express');
//create an app with express to handle http requests
var app = express();
//link to the pug views in the view folder
app.set('views', path.join(__dirname, 'views'));
app.set('view engine', 'pug');
//add cookie parser 
app.use(cookieParser());
app.use(express.static(path.join(__dirname, 'public')));
//views are setup in between
//socket setup
var socket = require('socket.io');
var server = app.listen(4000, function() {
  
});
//socket connection and some handlers
var io = socket(server);
io.on('connection', function(socket) {
 console.log('made socket connection', socket.id);
socket.on('users', function(data) {
  var response_data = {'username': data.username, 'id': socket.id};
  io.sockets.emit('users', response_data);
 });
socket.on('disconnect', function (data) { 
  console.log('disconnected user ', socket.id);
  
  io.sockets.emit('closed', {'id': socket.id});
 });
...
});

The last few lines show how you connect to sockets then on the server and for users handle an incoming message with a username and pass it on with that user’s socket session id.

On the front-end this is some few lines that show how the connection is made and handled.

//make socket connection
var socket = io.connect('http://localhost:4000');
//Listen for socket for `users`
socket.on('users', function(data) {
    var current_user_id = $.cookie('user-id')
if (data.id == current_user_id) {
        addUser(data);
    } else {
        $.cookie('user-id', data.id);
        $.cookie(data.id, data.username);
    }
let current_username = $('.sp-username').text();
//send username to `hello` so all logged in users broadcast their
//username to add to the list
socket.emit('hello', {
  username: current_username
 });
});

so socket on is for listening and emit for sending a message back. Once I got this running I wanted to also make sure I can tell when a user’s socket session is active or offline

I manage this on the backend by handling a disconnect event and broadcasting the user’s socket id, which is linked as an attribute on the liso it’s class is switched on the front-end and marked as inactive.

//backend
socket.on('disconnect', function (data) { 
  console.log('disconnected user ', socket.id);
  
  io.sockets.emit('closed', {'id': socket.id})
});
//frontend
socket.on('closed', function(data) {
     $(`.user-list li[unique_id="${data.id}"] i`).removeClass('online').addClass('offline');
});

I was also curious if I could use cookies to simulate a complete login since at a point I started to experience a bug with using sqlite and sequelize as the ORM to manage a database. It would create a database but when I tried to check in the user model if a user with a certain username exists it would crash with an unfamiliar error. Since I really didn’t need a database to use sockets I pushed that off for later. It’s possible to use cookies to login the username by “simulating” a login but to get it to work on different tabs on the same browser without using incognito was not possible “otherwise every tab will be the same user since they’re all checking the same cookie” you’d need a unique ID per session to be used as the user id and it can be then used to query for the username, something like this.

$.cookie('user-id', data.id); //from the socket.id
$.cookie(data.id, data.username); \\username also saved

But the issue with this approach also is on every reload NodeJS assigns a new socket ID for the user, that’s actually not a bug in the way sockets work since the javascript is reloaded and parsed again unlike a Service Worker that registers itself on the first load. Alternatively I will in future look for a way in node to get a session “and session id” which should persist throughout reloads and hopefully also like Django I can assign each session it’s own details on the server. So if you reload you won’t need to put in your username again to login. Because of the new session id on reload you’re user will be marked as inactive/offline on another opened tab until you login again.

At this point to get some more interactivity I also worked on “Log out” which would broadcast to all socket session you’re id which is enough to remove you’re username from the list. If you also type in the textarea on another opened tab you get user is typing message using jquery to bind to keypresson the textarea and this will broadcast you’re username to every other session except yours and that way using a list of usernames broadcasted and a join method with a space and comma if more than one user is typing you get user1, user2 is typing with something like this “though since onfocusout will remove the focus you can’t simulate more than one user typing easily, if you undo the focusout you’ll atleast get to see that part in action”

//handle adding the username to the list on broadcast received
var users_typing = [];
    socket.on('typing', function(data) {
     if (users_typing.includes(data.username) == false) {
      users_typing.push(data.username);
     }
if (users_typing.length > 0) {
      $('.typing').text(`${users_typing.join(", ")} is typing`);
     } else {
      $('.typing').text('');
     }
});
//handle removing username from the list when the user losses 
//focus on the textarea
socket.on('end-typing', function(data) {
     users_typing.pop(data.username);
if (users_typing.length > 0) {
      $('.typing').text(`${users_typing.join(", ")} is typing`);
     } else {
      $('.typing').text('');
     }
    });

and also as a way to remove this message when a user is removed from the list and if the list has no users don’t display anything by setting the text no an empty string. Bootstrap conveniently makes the badge invisible if it has no content.

Preview of typing message

That’s my experience so far with NodeJS, hope you enjoyed and learnt something. The repo to this project can be found here and likely will be active for a while.