Web Technologies for Musicians

or, From Score to Performance for Programmers

In this post I hope to give musicians (or anyone with a musical background) an easy insight into how a web page gets from the “internet” onto your browser. You won’t learn how to program, or even get a remotely detailed look at it, since this will be a high-level overview. But hopefully, it will allow you to be able to communicate more clearly with someone who is a programmer (especially if they’re working on or maintaining your site). And besides, we live in a world now where everything is basically a website; even many of the “apps” on your phone are just basically running a web browser.

For programmers who are interested in the behind-the-scenes for musicians and performers, this is just as simplified (and agenda-ified) a view of musicians as it is of web rendering, but hopefully it inspires you to read more into it.

Scenario: musician

Okay, so let’s say you’re a pianist – you’ve gone through years of school, learned how to interpret the score and make stylistically informed performances. Your teachers have put good habits into your practice and your playing, and you’re out on your own now.

You get a request one day to perform this cool Cage piece. Since you don’t have a copy of this piece, you have to get one from the library. You look up the phone number of the library, (123) 456-789 and call them, because you’re too lazy to go there physically. You ask the librarian if you can borrow the cool Cage piece. She checks her computer for a bit, and then puts you on hold. Moments later, she returns and says, “I’ve found the piece you wanted.” You’ll get it in a few days.

Few days later, the score comes in the mail. You take a quick look at the score. There’s a note in the first page that says that there are two inserts necessary to perform the piece. Confused, you call the library back, and ask the librarian about the inserts. “Ah,” she says, and puts you on hold again. She returns and says, “I’ve found the two inserts. One of them has some extra dynamics and articulations that either aren’t in the score or are corrections, listed by measure and beat numbers; the second insert has some complicated performance directions. We’ll send it over.”

A few more days pass, and now you have all the parts and begin learning the piece.

As you read through the piece, you realize that the first insert (of dynamics and articulations) has two options. One of them is meant for performances in large halls, the other for small halls. Furthermore, there is a small addendum in case the performer does not have access to electronic amplification.

The second insert is more involved. You have to roll dice to get random numbers, rearrange parts of the score, repeat some sections, etc. This is some serious business!

The day comes for your performance, and after all of your work reading the score, incorporating the right option in the first insert (you didn’t know how big the hall was, or if they could provide amplification, so you were pretty stressed out), and following all of the instructions in the second insert to a tee, you’re met with thunderous applause and standing ovation perplexed faces, furrowed eyebrows, and noncommittal applause, because modern music.

Scenario 2: Web Client (browser)

Okay, so let’s say you’re a browser, maybe Google Chrome – you’ve been programmed by thousands of programmers, learned how to read HTML, and render it onto the screen according to all of the specifications. Your programmers made sure you followed all the rules before they released you, and you’re now installed onto some bozo’s computer.

One day, the bozo opens you up, and directs you to go to a website. Since you don’t have this website in your cache, you have to call its host and get a copy. You look up the IP number of the library, 123.456.789.012, and call them, because you’re a browser, and you can’t go anywhere. You ask the host if you can get that HTML page that the bozo wants. She checks herself (because she IS the server) for a bit, and then puts you on hold. Milliseconds later, she returns and says, “I’ve found the HTML page you wanted.” You’ll get it in a few milliseconds.

Few milliseconds later, the HTML comes in through HTTP. You take a quick look at the page. There’s a note in the header that says that there are two inserts necessary to render the page. Confused, you call the host back, and ask it about the inserts. “Ah,” she says, and puts you on hold again. She returns and says, “I’ve found the two inserts. One of them is a CSS that has some extra styles that aren’t in the HTML, or are corrections, listed by element class or ID; the second insert is a JavaScript file. We’ll send it over.”

A few more milliseconds pass, and now you have all of the assets and begin rendering the page.

As you parse the HTML, CSS, and scripts, you realize that the CSS has two options. One of them is meant for large screens, the other for small ones like smart phones. Furthermore, there is a small addendum in case the browser does not support modern CSS grid-based layout.

The second insert, the script is more involved. You have to generate random numbers, rearrange parts of the DOM, repeat some elements, etc. This is some serious business!

The time comes for your rendering, and after all of your work parsing the HTML, incorporating the right option in the CSS, and following all of the JavaScript instructions to a tee, you’re met with gratitude and praise from the bozo complaints that it took too long to render, and questions of why the scrolling is so laggy.

The Components

HTML <-> Score

HTML stands for HyperText Markup Language. Hypertext just means text that can contain hyperlinks (which are links to other webpages). The important thing is that it’s a markup language which is a way to create documents or pages that describes not only the content, but any other metadata attached to it. There are many other markup languages, like XML, EPUB, and RSS.

Music notation is a markup language. It contains not only the raw content (which we can call the notes), but also metadata, including time signatures, tempo markings, key signatures, ossia, etc. In fact, this is why there is a portable music notation format, MusicXML. Most importantly, without scripts, HTML is static, unchanging. Scores, without performance indications, remain static as well.

Just like you can define things like a <table>, <div>, or <script> in HTML, you can create <measure>, <clef>, and <pitch> in MusicXML. In most markup languages, you need to close your tags with a corresponding one: <table>content</table> or

<pitch>
    <step>C</step>
    <octave>4</octave>
</pitch>

CSS <-> Dynamics, articulations, etc.

CSS stands for Cascading Style Sheets, which contain the styles necessary for rendering the page. Styles can also be defined in the HTML, but for reasons such as the one in the scenario above, you might need different style sheets depending on the window size, or even the font size. It’s called cascading, because styles defined later override previous ones, just like corrections in scores. The difference in this comparison would be that we usually don’t put dynamics and articulations in a separate file, again, unless they’re corrections.

JavaScript <-> Performance directions

JavaScript is a programming language. Browsers can read these scripts and perform the work that is asked of them. For example, a browser could be asked to fetch a video, or calculate some numbers, or render 3D graphics, just like how performers can be asked to sing, hit the inside of the piano, or sit there in silence. These are not standard things in music, and equivalently, the things that JavaScript asks the browser to do are not able to be represented in HTML, because they often encode dynamic behavior.

In modern websites, most of the power and heavy-lifting is in the JavaScript. When you see a website (like mine) that is a SPA (single-page application, which means that clicking on different links within the site causes the content to be changed dynamically, and not through an entire page load) you can be sure that most of the work is being done by JavaScript. In fact, my HTML file is probably 30 lines long, whereas there are tens of thousands of lines of JavaScript. By using JavaScript, you can generate HTML on the fly, and so the user experience is more interactive and dynamic.

Programmer <-> Composer

In this case, we are talking about a front-end programmer. They program the HTML (in a very generic sense, keeping this high-level), just like composers write down the score.

If we are to make it more modern, programmers usually work indirectly with HTML, usually writing code that generates HTML (either on the server or client side). The equivalent would be a composer jotting down sketches, and then having an arranger flush out the orchestration, or a publisher cleaning everything up and typesetting it so that it is readable.

Client (Browser) <-> Musician

Musicians are trained and have musical knowledge within them to read the score, and parse the performance indications. Browsers each have a rendering engine – Chrome and Safari use versions of WebKit, Firefox currently uses Gecko, and Edge uses EdgeHTML. These all read HTML and can translate that into a web page, just like how musicians can read a score and turn it into music. Furthermore, all of the browsers have a JavaScript engine – Chrome uses V8, Firefox uses SpiderMonkey, and Edge uses ChakraCore. (The one clear takeaway is that programmers need to come up with better names.)

Server <-> Library/Music Store

To get the score we have to either get it from the library or a music store, if we don’t already have it. Similarly, the browser needs to get the HTML page from a server, if they don’t already have it in the cache. The server does a lot of things behind the scenes, just like a music store or library would have a lot of internal workings that you don’t see, such as how different pages are stored, how databases are accessed, etc.

 

I hope this gives you a bit of an understanding into what goes into rendering a page. I didn’t define every term in the scenarios, but I think given the parallel, they should be self-explanatory. And, if not, there is always Wikipedia. Let me know in the comments if there’s anything specific you’d like to hear about!

Things I Learned Making a Website from Scratch

I think I’ve reached a point with this new website where I feel comfortable saying it is pretty much done. I just wanted to do a quick post about the process, and what I learned. First off, all of the code is open source on GitHub: https://github.com/kamiyo/sycpiano. Thanks also to my brothers Jason and Andrew for getting me started and helping me with things, especially in the earlier stages. It took around 2 years! Don’t worry if the acronyms mean nothing to you – I’m going to do a simple musicians guide to web programming later!

Feel free to skip to the last section (Main Takeaways) if you want to read the main takeaways. The following sections are just me bragging a little bit about the web programming technical things I learned, so I can feel a bit better about myself and not like I’ve wasted two years in working on this website…

Going In

I had programmed my previous two sites myself as well, but I didn’t have the guidance of two programmers at those times. To go back further, I was messing around with HTML and CSS back in the blogger and Geocities days.

So I came into this process knowing a bit:

  • HTML
  • CSS
  • JavaScript (vanilla)
  • PHP (sorta)
  • MySQL (very basic)
  • Messing around with .htaccess

I was fortunate to have taken some comp sci courses when I was in school, so I had a solid foundation of any ALGOL-like languages (C, C++, Java).

What I Learned

The early stages of the website was all about learning the framework and modern JS. In my old site I just gzipped up the JavaScript file, and served it along with some libraries like jQuery through their CDNs.

My brothers suggested I use React as my framework, whereas for my previous site I used AngularJS (v1). They also setup a toolchain using Webpack and Gulp, running on NodeJS. Later on we dragged Typescript in.

So, things I learned:

  • Almost everything runs on JS now.
  • You basically never touch .html files anymore.
  • React
  • NodeJS
  • Reducers and state management
  • Typescript and Modern JavaScript (es6+)
  • Promises!
  • Using and creating a restful API
  • Using a build tool like Webpack
  • Postgres, using an ORM like Sequelize, and Full-Text Search
  • A bit about Nginx and some server stuff like using pm2 to keep the Node server running.
  • More familiarity with version control (git)
  • CSS-in-JS solutions
  • Canvas and WebGL animations
  • Using Chrome devtools

And not web-related, I learned a bit on using OpenCV and doing some facial recognition (trying to automate thumbnail creation of my own photos) and about constant-Q transforms when turning audio time data to frequency data (as seen in the music visualization). I’ll also do a post about that.

Main Takeaways

Mostly, I learned I have no design chops. I have the hardest time choosing colors and layout, and making user interaction decisions. So much of the design is literally hours of trial and error. The coding part is fine. It’s the design part that is hard.

It takes time. All the code planning, debugging, refactoring, especially when you don’t have experience working with a large codebase. But you learn alot. And you save money.

Everything is on the internet. Stackoverflow, GitHub issues and gists – if you have a problem, chances are someone else has had it before you.

Cross-platform is a b*tch. Just because it works on one browser on one device, doesn’t mean it will work on another one. I’m also starting to think mobile-first would be the best way to go (which is not what I did). It’s harder to do good mobile UI/UX compared to desktop. You can tell on my site mobile was (not-quite, but almost) an afterthought… or like a halfway-through-thought.

Designing your own website means you can have it exactly the way you want. If you want a feature, you implement it. If you can’t, it’s because you’re not good enough, not because it’s not possible.

Modern web development seems to be “libraries, libraries, libraries” but it really does make it easier. Because of that, a lot of development time is spent also on building and making sure the shipped code is compact and performant.

Finally, just like how artists learn by tracing, composers learn by transcribing, and musicians learn by listening, it’s helpful to start coding from examples. I was sort of reluctant to learn React, and all this new stuff, but since my brothers helped get things started, I could study what they had written and learn from there.

And just like music, do it only if you enjoy it.