Experiments with ReactJS

TL;DR: tried ReactJS, went back to d3js.

Web Components are on their way to a browser near us, and I’m not sure if I’m excited about them or not. On the one hand your HTML will be much easier to read and to write. On the other hand a lot of its functionality is already there in frameworks, under various names such as partials or helpers. For instance a “product” Web component could be written as:

  <product><!-- stuff --></product>

in ERB:

  < %= render partial: "products", collection: @products %>

Also, the shadow DOM is scary.

There is a polyfill you can use to play with Web components, but Reactjs also offers custom element functionality, and is being talked about a lot currently. So I spent a few days playing with it. It was interesting, but I didn’t really like it. That’s why you won’t find much react in this repository, except if you browse through its history.

Reactjs not only it brings custom elements, but also data synchronisation between backend data and the DOM.

React isn’t the first library to do it. Yet what it does really well in that it updates only the parts of the DOM that need refreshing, instead of rebuilding everything from the server. It’s fast, and saves you from doing it yourself, as I imagine it’s not trivial.

The other thing is JSX, which is what presumably confuses people the most about React. Your markup (some HTML and all your custom elements) is now in your JavaScript. You can write (From the tutorial):

var CommentBox = React.createClass({
  render: function() {
    return (
      <div className="commentBox">
        Hello, world! I am a CommentBox.
      </div>
    );
  }
});
React.render(
  <commentbox></commentbox>,
  document.getElementById('content')
);

Hmm. Maybe it’s alright. May it’s the future of web UI design and I’m not used to it yet. Maybe separating JS, HTML (and CSS) is too axiomatic and I’m too resistant to new paradigms. Let’s continue the tutorial. We come to:

render: function() {
  var commentNodes = this.props.data.map(function (comment) {
    return (
      <comment author={comment.author}>
        {comment.text}
      </comment>
    );
  });

A map to generate multiple components from data. Clever. Not the most readable, though. I think a for loop or a foreach would work better. But why not just augment JSX so you can write:

{foreach comment in this.props.data}
  <comment author={comment.author}>
    {comment.text}
  </comment>
{/foreach}

as is common in templating languages? And then you get all those messages in your javascript console about having to add keys to your custom elements, and when you look up how to do that you end up in the depths of the React documentation, which is confusing when you just want to do the tutorial. Anyway, let’s carry on.

The code I want to write is part of an application that has a dashboard which shows images on a timeline. I want that timeline to be SVG, with axes, animations, etc.

So here I go:

render: function() {
  var dayList = this.props.data.map(function (dayData) {
    return (
      <image xlink:href={dayData.thumbail} x={dayData.time} y="200"></image>
    );
  });
}

and there comes the killer console message:

error: Namespace attributes are not supported. ReactJSX is not XML.

Why can’t React just consider xlink:href as just an attribute names with a colon, I don’t know. That message is unapologetic and almost sounds as if React makes it an error on principle, because it doesn’t like namespaces.

But there is a way around it:

render: function() {
  var imageTag = '<image xlink:href="meh.jpg"></image>';
  return <svg dangerouslySetInnerHTML={{__html: imageTag }}></svg>;
}

Hacky and apparently dangerous. Not the most enticing feature of React.

At that point I’m giving up, thinking that React may well be a good idea but its execution isn’t ready yet. Hopefully it will be soon and we all learn to love JSX. Honestly, I think we won’t.

Now I’m generating my SVG using d3. I could have continued with React and gone for HTML with CSS3 transforms and all the things, but I’m more familiar with SVG+d3. So my code became:

 day.append('svg').attr('width', 1000).attr('height', 300)
    .selectAll('g')
    .data(function(d) { return d.events; })
    .enter()
    .append('image')
      .attr('xlink:href', function(d) { return d.href + '.jpg'; })
      .attr('x', function(d) { return d.time })
      .attr('y', 0)
      .attr('width', 50)
      .attr('height', 50);

Sure, the markup is still in the JavaScript code, yet the syntax is not as unsettling (especially for code editors) and is simpler and readable — not as much as with templating languages, admittedly.

True, you do need to get into the d3 way of doing the data mapping, but it’s worth it. The synchronisation isn’t as clever as React’s, either. But perhaps that will be improved, and to be fair it’s only useful for applications that do realtime updates, à la twitter or facebook.

From now on, if I want to keep investigating Web Components, I guess I’ll have to look at webcomponents.js, or keep Reactjs and forget SVG and stick to HTML, wait for native implementations or try and figure out a way to do d3 with templates.

Posted in General | Leave a comment

Narrative timelines

Historique is a new web application to make interactive narrative charts. It lets you create events with dates, people with pictures and displays everything on various timelines.

Click on the screenshot to see a narrative chart of Raiders of the Lost Ark.

Screen Shot 2014-06-14 at 20.01.41

It was originally created to visualise the story of the research institute where I studied. In order to demo the concept, I wrote a generic applications and worked on a couple of instances (Yesstory, and this one). Possible applications include:

  • family history (showing weddings, births, etc)
  • organisations (creations, milestones, employees)
  • novels, films (chapters, scenes, characters)
  • Bands (albums, concerts)

The technologies used include: Django, Postgres, D3.js, nginx, bootstrap, heroku.

Posted in in English, petits dessins, software | Leave a comment

Tabale, a meeting organiser with an IVR interface.

One of the deliverables of the VOICES project is a meeting organiser web application. With, unsurprisingly, database tables for users, meetings, participants, time slots, etc; nothing a competent hacker couldn’t write in a few days using any web app framework.

Except for one thing: the system has to be accessible to meeting participants through an interactive voice response (IVR) system: they can call, or be called, to hear about the meetings and say if they’re going to attend or not. And it has to work in multiple languages: French, Bambara and other languages from Mali, where the system is currently deployed for Sahel Eco, a local NGO that works to improve the lives of people living in rural areas. That instance of the application has users that speak multiple languages, meetings and announcements that can also be in various languages. Everything has to work so that users (and administrators) can use the system in their favourite language, or at least one they understand.

The application is called Tabale (a popular musical instrument, shown left). Although it uses a lot of different technologies one the web-facing and server sides (I especially liked using RedbeanPHP, Handlebars), the interesting part is the IVR, written in VoiceXML and running on the VoiceGlue+Asterisk setup offered by Orange’s Emerginov Platform. It’s not because phone calls are linear and have low information bandwidth (compared to graphical or textual interfaces) that writing IVR applications is easy. Especially for an audience that has never encountered speech applications before and only use their phone to talk to humans. For instance, the application developer has to keep in mind that timing is essential: the user will be switching from putting the phone to their ear, to listen or speak, and in front of their eyes to type dial tones. They will have to know when to speak to record a message, and how to end the recording. And there are many other rules that, if not followed, will make your IVR very irritating: menus (like “press 1 for listening to the meeting announcements, 2 for leaving a message, …“) shouldn’t be have more than a few items. Or you shouldn’t have your system say “press 1 for …, 2 for …” but instead “for …, press 1. for …, press 2“. Etc.

Language support has to be done right, too. The first time a user calls, they should be asked what language they want to use, and remember that choice for subsequent calls. A meeting announcement should be played in that language, and if it’s not available there should be a fallback language that’s likely to be understood by most. And by far the hardest challenge was debugging the IVR. VoiceGlue’s support for VoiceXML is random and not very well specified, and the Emerginov platform doesn’t provide call logs. Writing unit tests is also very hard, if you want to go beyond simply testing if your PHP-generated VoiceXML files are valid.

As the VOICES project comes to an end and Tabale is pretty much complete, I’m releasing the source on github. I might deploy a test instance somewhere using Evolution for the telephony features, although I wouldn’t want people to use the system to spam others with automated phone calls. But I’m more than happy to help anybody interested in deploying it for themselves or their own organisation.

Posted in General | Leave a comment

Simple street maps in SVG

OpenStreetMap Brussels

Simplified (click to enlarge)

As part of some data visualisation I’m currently working on (more on that soon), I needed a way to show simple street maps in the background of the data to be visualised. I looked around for existing SVG map renderers, but didn’t find much that was easy enough (to be honest I didn’t look that much, as I thought it would be a fun thing to do myself, and I’d already done something very similar with Maporizer).

The map data come from OpenStreetMaps (using their custom XML format), and I’ve used XSLT to transform them into SVG.

Everything is available on github. The README file explains what needs to be done to obtain the maps from OpenStreetMap and run the converter.

Notes:

  • It’s not very hard. XSLT makes XML-to-XML transformations easier to write than any other language.
  • What’s tricky is that, on large scales, the osm files can be huge. The osm map of Brussels as shown on the image above is a 110MB file, for instance. That’s why there are two XSLT transforms: one to select the elements that should be shown on the SVG, and another to transforms those elements into SVG markup
  • The params.xml file shows how to simply select the map elements to select. And the styling in the SVG is done using a CSS stylesheet whose classes correspond to the types of elements shown on the SVG. Both are straightforward, although you have to know a little bit of how openstreetmap names map elements (like ‘primary highway’)

For now I don’t need anything much more complicated, but I believe that without much effort it’s possible to render more map elements, like buildings, street names, etc. One would just need to select the element to be added in params.xml and its appearance in style.css.

Posted in General | Leave a comment

Camera Capture Works Wonders

I’m pondering whether to write a textorizer app. (I know I should stop beating that dead horse, but I still get feedback about it. It’s a good testbed, too). It would be nice to be able to have it on your mobile device, take a picture, see the results, share them, make money, etc.

However I remain more interested in the possibility of continuing to run it as an HTML5 app. But right now, the missing elements are:

  • support for the File API in some of the mobile browsers
  • offline: I need to investigate that. Saving a page “to read later” in the Android browser doesn’t save the JavaScript code, but it may be possible otherwise.
  • camera capture. This one’s no longer missing, see below.
  • make the page itself mobile-friendly (not a missing feature, just something I need to do).
  • no easy way to sell the app

Indeed, if you take your Android device, go to textorizer, and click on “select file”, it will now offer to take a picture (or select one from the device). I just tried it on my phone and it was jolly good fun. It worked with the default Android browser, Chrome and Firefox (shown below).

The only change required was to add “capture=camera” on the file upload control:

<input id="file_selector" type="file" accept="image/*;capture=camera" class="image_selector"/>

I didn’t expect it would work when you don’t actually upload a file but instead retrieve it in JavaScript, but in fact it does work, and we’re one step closer to webapps everywhere. I know it’s not yet a WHATW3CG standard, but it’s still a step forward.

Posted in General | Leave a comment

The trouble with VoiceXML (part 2)

(continued from part 1)

VoiceXML remains to this day one of the most successful standards to come out of W3C. Even though the Voice Browser Working Group, the committee that designed it, never got the same visibility as the ones that designed CSS or XML, it continues to be one of the biggest working groups at W3C. Most of the IVR industry is a member, and it has managed to produce a true industry standard. The working is currently designing VoiceXML 3.0. However, VoiceXML will probably never enjoy the success it once had.

With online access becoming ubiquitous, people don’t call IVRs so much anymore. It has become far easier to order pizza or book a flight online, rather than to run through the lenghtier process of doing it on the phone. In fact, there is not that much innovation in the area of IVR applications any more (maybe better speech recognition allowing mixed initiative dialogue, but that’s rarely found), and the the most popular applications like voicemail or simple menu-based information services, are sufficiently stable and familiar to users that there is no need for real innovation (this is not true everywhere, though — see below). You don’t find so many job ads for IVR designers these days, and there hasn’t been a book published on VoiceXML since 2002.

That does not mean that voice applications aren’t dead, though. Open source telephony software is going strong with Asterisk or Freeswitch, originally PABXs, but which also integrate IVR functionality through proprietary scripting languages. Moreover, companies like Voxeo and Twillio offer cloud services for developing Speech and SMS apps, and are enjoying a certain amount of success. Interestingly, Voxeo offers two ways of designing applications: using VoiceXML, or with an API for various standard languages (PHP, Python, etc.). The latter is much more used, because there are many more hackers familiar with those languages than there are who prefer VoiceXML. The programmatic approach especially makes it easy to integrate voice in other Web applications, make mashups and everything Web 2.0. Even though VoiceXML 2.1 extended the language specifically for that purpose, it hardly changed things. Another recent evolution of speech applications is that they are now found on mobile devices: iOS’s Siri and Android’s Voice Search are the best known examples. Those applications aren’t written in VoiceXML either, also because they aren’t just based on voice, but combined with visual and tactile interaction.

VoiceXML is catching up, though. Version 3 will handle multimodal interaction, for instance. And the working group is working on complementary standards, such as SCXML, which can declaratively describe interactive applications independently on whether the application is visual, audio, or a combination or both. But until those standards are finished and implementations are available, the procedural approach offered by Tropo, Android, or Microsoft is gaining in popularity. Moreover, standardisation is also happening in that area too: the W3C’s HTML Speech Incubator Group and Speech API community group are very active in defining a speech API for web page, some of which is already implemented in browsers.

All in all, it will take three things to make VoiceXML return: the working group finishing version 3, implementations being released publicly (breaking the monopoly of mobile operators and their walled gardens), and VoiceXML supporters convincing developers that designing elaborate speech applications leads to extremely complex code, and can only be avoided using declarative markup.

 

Note: the above currently doesn’t apply to developing countries, by the way. Voice-based web access in Africa is at the heart of 5 projects that the Web Foundation (my employer) is running, and all the IVR we design use VoiceXML. For various reasons (illiteracy, cost of smartphones, culture) the voice channel, as opposed to SMS or internet access, remains one of the most important, for human-to-human communication but also human-to-machine. That means there is a great potential for voice-only online applications that’s yet to be tapped into. Many projects coming out of the Foundation’s “mobile entrepreneurship labs” are signs of that potential. Yet there is little doubt that, with the rise of the mobile web in Africa, the voice channel will eventually follow the trend described above.

Posted in General | Leave a comment

Typographic maps using OpenStreetMap, XSLT, JavaScript and SVG

Recently I came across Typographic Maps, and I thought I’d try something similar. Not by hand, like they do. Instead, grabbing some map data off OpenStreetMap and automatically transforming it into SVG, as shown below. Not as nice as the original, but I didn’t want to spend hours making it look better. (It would probably never look as nice as when done by hand, although the ability to choose any place is a plus.) Instead I learned a few interesting things along the way.

Brussels Centre (SVG version)

How it works:

A rectangular map is selected. I used GetLatLon to find the exact latitude and longitude of the test maps.

The OpenStreetMap data is retrieved (here, Hyde Park Corner):

http://api.openstreetmap.org/api/0.6/map?bbox=-0.1593017578125,51.49805708407405,-0.14591217041015625,51.50687269909403

This returns nice and well-formed XML.

The format is easy to understand, and so it was also easy to write a basic XSLT stylesheet to select features of interest (roads, parks, etc) and turn them into SVG. Again, if someone ever wanted to make it into a generic tool, there should be a GUI lets you choose your map and the tool would automatically adjust to the scale and filter out details that are too small and make the output a mess like now.

Here is the rendering of Hyde Park Corner (and in SVG):

Hyde Park Corner

And that’s it.

Now what’s really interesting is that while looking for the latest version of Saxon, I discovered SaxonCE, a JavaScript version of Saxon. It’s not even a wrapper for the Java version, it’s the actual Saxon code cross-compiled to JS (using GWT) and running in the browser. Very impressive. And so, you can now bypass your browser’s old and buggy XSLT engine and instead add to an HTML page something like:

<script type="text/javascript"
        language="javascript" 
        src="../Saxonce/Saxonce.nocache.js"></script>

(You can also use the old XML Stylesheet Processing Instruction, and there’s a JS API, too). Here’s an example using the <script> construct above, where the XSLT transform mentioned above inserts the SVG map into an HTML page, where the <script> is. It takes a while to run, but it works, and in all browsers I’ve tried. When I think about the fact that the XSLT transform itself contains some JavaScript to text nodes to the SVG DOM, my head starts spinning.

All the code and examples are at github/maxf/maporizer

Posted in General | Leave a comment

iOS Celtic

Earlier this month, jwz announced he’d finally got Apple to approve the iOS version of XScreenSaver. Koalie was very kind to send me these screenshots of my “celtic” hack running on her iPhone.

After Witali Aswolinskiy’s iTextorizer, that’s the second of my graphics hacks ported to iOS. Saves me doing it! A big thank you to their respective authors.

Posted in General | Tagged , , | Leave a comment

The trouble with VoiceXML (part 1)

Following up on the previous entry I thought I talk about more technical details on how, at the Web Foundation, we’re designing our radio-platform.

In general, voice application share the same architecture as standard websites. Just replace “browser” with “voice browser” and “HTML” with “VoiceXML” (the most widespread language for voice applications). Also don’t put the browser on the user’s computer but on the web, usually not where the application server is since it’s often provided by a third-party, like a telco.

Voice apps vs Web apps

Because VoiceXML is the HTML of Interactive Voice Response applications you can do just as you would in a standard web application and generate the files served using PHP.

Here’s a basic (simplified) VoiceXML file:

<vxml>
  <form>
    <field name="year">
      <prompt>Please say the year you were born</prompt>
      <grammar src="year.srgs"/>
      <noinput>You did not say anything</noinput>
      <nomatch>I did not understand</noinput>
      <filled>
        <if cond="year &gt; 1980">
          <submit next="senior.vxml.php" namelist="year"/>
        <else/>
          <submit next="senior.vxml.php" namelist="year"/>
        </if>
      </filled>
    </field>
  </form>
</vxml>

Unsurprisingly there is, unlike standard HTML, some logic in the application. In fact a large portion of the VoiceXML specification describes the Form Interpretation Algorithm, which goes far beyond simple <if> statements, but includes features like error recovery, events and exceptions. Things that are barely visible in the language’s syntax, but are rather complex. Barely visible, that is, when you’re writing simple examples. But in a real application, things becomes quite complex and the resulting VoiceXML files can be hard to read (a bit like XSLT).

And you can add to that the complexity of PHP, because server-side logic is mandatory. Indeed, a VoiceXML application being just a set of forms, each one has to <submit> its contents back to the server, which then generates and serves the next VoiceXML file.

And little by little you end up with code like what I put at the end of this post. What was originally a simple VoiceXML file has become a horrible mix of two languages. Despite the ugliness it’s still code that looks familiar to many PHP developers. But again, this isn’t just PHP generating HTML, this is PHP generating VoiceXML, itself a programming language. (Yes, HTML can also contain JavaScript. Guess what, so can VoiceXML).

I’m not the first to notice it. In 2007 the W3C’s Voice Browser Working Group released VoiceXML 2.1, which adds a small number of features that can help us, the <data> tag, which lets you do XMLHttpRequest stuff, and <foreach> to loop over a variable. <data> is great, because instead of having to submit a form back to the server and receive another VoiceXML file, you can send the data over but remain in the same file. And <foreach> also removes some dependency on server-side logic. However, I know of no VoiceXML browser that implements the specification completely, including the one I’m stuck with (Voice Glue). Seven years after the release of the specification.

Are things going to improve? Are implementations going to catch up, especially FOSS ones? Unlikely. For the reason that VoiceXML is dying. I’ll write about it, and the present and future of voice applications, in another entry.


And now the ugly code (which is not too bad, actually, but you can see how it quickly gets much uglier). Nothing but code-generating code; imagine the debugging, especially when all the error reporting you have from the VoiceXML interpreter is a message on the phone saying “A serious error has occurred. Exiting.”

<?php
// authorization: get callerId, try and match it against the user list
// if it checks, go ahead. If it doesn't, create a new user
// input variables: callerId

require_once('log.php');
require_once('i18n.php');
require_once('radio-platform.php');
require_once("ivr-platform.php");

Log::write("starting auth-callerId");
Log::write($_SERVER['REQUEST_URI']);

if (isset($_REQUEST['callerId'])) {
  $callerId = $_REQUEST['callerId'];
} else {
  $callerId = 'unknown';
}

$sessionId = $_REQUEST['sessionId'];

// fetch user list
$users = RadioPlatform::getUsers();

// search user with correct callerId
$userFound = false;
foreach ($users as $user) {
  if (phoneNumbersMatch($user['phone'], $callerId)) {
    $userFound = $user;
    $userId = $user['id'];
    $userRadioId = $userFound['radios'][0];
    break;
  }
}

if ($userFound) {
  $userLang = $userFound['lang'][0];
  Log::write("User: $userId");
} else {
  Log::write("No user found.");
}

header('Content-Type: application/voicexml+xml; charset=utf-8');
print('<?xml version="1.0" encoding="utf-8"?>');
?>

<vxml xmlns="http://www.w3.org/2001/vxml" version="2.1">
  <property name="inputmodes" value="dtmf"/>
  <var name="sessionId" expr="'<?php echo $sessionId ?>'"/>

<?php
if($userFound) {
  $radios = RadioPlatform::getRadios();
?>
<form>
  <var name="userId" expr="'<?php echo $userId ?>'"/>
  <var name="userRadioId" expr="'<?php echo $userRadioId ?>'"/>
  <var name="userLang" expr="'<?php echo $userLang ?>'"/>
  <block>
<?php prompt($userLang, 'welcome') ?>
    <audio src="<?php echo $radios[$userRadioId]['audio']?>"/>
    <submit next="main-menu.vxml.php" method="get" namelist="userLang userId userRadioId sessionId"/>
  </block>
</form>

<?php
} else { // No user found through callerID. Create new user.
?>

<form>
  <block>
    <?php prompt('bam','welcome'); ?>
    <?php prompt('fr','welcome'); ?>
  </block>
  <field name="userLang">
    <?php prompt('bam','select_bam_1'); ?>
    <?php prompt('fr','select_fr_2'); ?>
    <option dtmf="1" value="bam">Bambara</option>
    <option dtmf="2" value="fr">French</option>
    <noinput><reprompt/></noinput>
    <nomatch><reprompt/></nomatch>
    <filled>
      <var name="callerId" expr="'<?php echo $callerId ?>'"/>
      <submit next="auth-new.vxml.php" namelist="userLang callerId sessionId"/>
    </filled>
  </field>
</form>

<?php } ?>
</vxml>

<?php
// tries to fix bad callerIds, removing leading whitespace, '+' or '0'
function clean_phone_id($caller_id) {
  $ph=ltrim($caller_id);
  $ph=preg_replace('/\s*$/','',$ph);
  $ph=preg_replace('/^\s*/','',$ph);
  $ph=preg_replace('/^\+/','',$ph);
  $ph=preg_replace('/^0*/','',$ph);
  return $ph;
}
// returns true if both numbers match
function phoneNumbersMatch($n1, $n2) {
  if ($n1 === $n2) return true;
  return clean_phone_id($n1) === clean_phone_id($n2);
}
function prompt($lang,$msg) {
  $xmllang = IvrPlatform::xmllang($lang);
  echo "<prompt xml:lang='$xmllang'>".I18N::say($lang,$msg)."</prompt>\n";
}
?>
Posted in General | 3 Comments

Foroba Blon and our Radio Platform for Citizen Journalists

Radio Moutian in Tominian, Mali

This is a re-blog from an entry I wrote for the Web Foundation, my current employer. Not something I usually do, but I really care about the project and I’d like it to be more known. Not that this blog makes a difference, but who knows.

This week, a group of us are back in Mali, on a field trip for the Foroba Blon project, where we’ll be deploying the first prototype of our radio platform, testing it, and gathering feedback for the continuation of the project.

When I describe the platform and the project I often get the same questions, so I thought I’d write an FAQ.

Continue reading

Posted in General | 1 Comment