We’ve got several CDs of Khaled. I like them for his voice, and the swing and rhythm of the music. Much of it is very “dancable”, but at the same time the rhythms are more than the simple ONE-two-THREE-four of western pop.

So I thought I’d really enjoy him in concert this evening. I enjoyed it so little that it made me wonder about things.

#1: The setup.
On the CDs he’s often accompanied by only two or three instruments – acoustic guitar + accordeon for example, or drums + violin + piano. His voice gets a lot of space, and has a lot of depth.
Today, he was backed by lute, base guitar, 2 electric guitars, one whole rock-style drum set, one hand drum, two keyboards, and a 3-man brass section. His voice had two layers of effects (vibratos and echo) and during some songs, one of the keyboard players was doing more singing than Khaled himself. The net effect was that his singing got blended into a general mass of sound and didn’t stand out, and it all sounded more like a standard rock concert than rai.

Does his voice no longer work on its own – has he lost it? Or is this an attempt to capture larger Western audiences by adapting the style to what the average European is used to?

#2: The lighting.
A floodlight of pure white, aimed at the faces of the audience, and about 4 times larger and stronger than anything aimed at the stage. Not just a little spotlight, this was so bright that it made my eyes water even when I closed them; I had to block it with my hand. What does a lighting designer think when doing something like this? “Let’s weed out the weak ones?”

#3: The volume.
Start out somewhat loud-ish. Turn it up. (We don earplugs.) Turn it up some more. And then a little bit more. Until it got to the point where it we found it physically painful, couldn’t stand it any more, and walked out.

This was even more of a surprise because the Barbican can usually be relied on to provide good (or at least reasonable) sound quality – unlike the South Bank Centre (Royal Festival Hall / Queen Elizabeth Hall) that we’ve stopped going to for concerts, because their sound been bad far more often than good.

This is not the first time we leave a concert because it actually hurts, so we’ve asked ourselves the same questions before.
How can everybody stay there and seem to enjoy it? Are they all half deaf, since they’ve been hearing music at this volume for years? Do they hear but don’t mind?

And more importantly, why is it done this way? Do people like it? Are the sound engineers deaf themselves? Or does everybody in the audience have tiny tinny speakers at home, so that they don’t know what music sounds like when it’s good – when the sound is well balanced and the volume is appropriately loud?

So I Googled for a bit (“concert too loud”). The most informative page I found was Edward Tufte commenting on the same issue on his web site (which has a whole lot of other interesting stuff too). Here are some of the responses:

The stage foldback (or monitor) system is independent of the main sound system and creates an intentionally different mix (often a separate one for each member of the band). The level is often extremely high to get control of the mix (eg if you have a double Marshall stack right next to you, the vocals in the foldback have to be loud enough to get above the guitar level). This does mean the house system (the audience’s) has to be loud enough to get above any ‘spill’ from the foldback system.

I had an interaction with a sound engineer setting up a performance. I expressed my concern over the high sound levels. He reassured me that his group had found that if the levels started low and then gradually increased, the congregation is not aware of the high levels of exposure.

In my rock club experiences, the sound engineer is typically the deafest person in the room. The engineers have subjected themselves to more loud music over the years than even the band members since many of them are “house” engineers or, if touring with bands, are out in front of the band night after night, soaking up the decibels. The ubiquity of “treble creep” is overcompensation caused by hearing that is literally notched out by damage in the higher tonal ranges. This explains the excruciating sharpness so common in live rock audio mixes these days.

I think another factor here is key, the specious practice of amplifying the drum kit. I think this got started when rock bands began playing arenas, but it then became fashionable to do this in even the most intimate of clubs. For anything but the most expansive club, the typical rock drummer is already playing at ear-splitting levels without any amplification whatsoever. Amping it just makes it worse, and a byproduct is that all the other instruments have to turn up to compete.

And a related comment regarding sound quality (from a standup comedian):

It is harder to be funny in a room with a very high ceiling – because the all-important start-up laughter from a small part of the audience has little contagion effect with the rest of the audience. The start-up laughter at a remark takes several seconds to go up to the high ceiling and come back down, too faint and too late to reach the yet-to-be amused members of the audience. The Comedy Connection has a low ceiling for good reason.

All quite interesting. I think the only conclusion from this is that in the future I will think twice before buying tickets for a concert by one of the big-name artists. The less mass-market ones are likely to care more about sound quality.

Today, we went home and enjoyed Khaled on CD instead.

We went to a concert earlier this evening – Syrian Sufi music with whirling dervishes. One lead singer, a flute, a zither, a percussionist, two whirlers, and 3 background vocalists. The lead singer was good and had a pleasant voice, and the zither player was very skilled. Some parts of the show were a bit coarse and rough – there was a tendency for the sounds to become indistinct, as the group gravitated towards the “more! louder! faster!” school of music, where a more subtle and nuanced style would have suited better. (They played some taped sufi music after the show, as people were leaving the venue, and the difference in quality was clear: the taped music had a heartbeat-like natural pulse, where this evening’s performers had a harsher feel.) Even so, the concert as a whole was pleasing.

I enjoy many sorts of religious music – gospel, hymns, Russian and Greek orthodox and Gregorian chants, qawwali etc. Music has a different quality when it has a soul, and musicians sound slightly different when they sing/play for their belief and not for money – their commitment and devotion shine through. And I like the calm and weighty feel of chanting.

(The concert venue was, quite fittingly, a converted church: LSO St. Luke’s, in Old Street. It’s a nice space, simple and open. The nave has been opened up completely, and a balcony built along the north and south sides; the stage is towards the East. The walls are bare, and all the large windows with many small panes have been left in place, although they’ve been covered up for some concerts. It should look great from the outside at night.)

I wonder what an Arabic-speaking Muslim would experience in a concert like this. I do not understand any of the content (calling it “lyrics” doesn’t seem entirely appropriate) and I only know purely factually that these are devotional chants. I cannot be part of it the same way as the dervishes are.

And I also wonder what music sounds and feels like in Eric’s head. I know it must be an experience that’s very different from mine. For me, the percussion-backed song / chanting was the best part of the performance – immersive, meditative, passionate. Eric on the other hand enjoyed the zither most, and I could see others in the audience agreeing with him. I found the zither too alien: even though I’ve heard fair amounts of non-Western music, it was too different from what I’m used to hearing, and its music kept slipping out of my grasp. There was nothing that I could recognise as melody or rhythm, and while it made a pleasant sound, I was unable to really appreciate it. Sort of like a babbling brook – pleasant but ultimately not interesting.

Interestingly, about a dozen people left early, at various points during the concert. Did they buy tickets without knowing what Middle Eastern Sufi music sounds like, and find it too strange?

This week has seen a lot of SAS work, so that’s what my head is full of right now.

• After several weeks of intensive SAS usage, I’m finally reaching what would be called “conversational” for a human language. (This came after a lull of over a month where I was mostly working in Excel and VBA). I can express any basic concept, and no longer need to search for words every time I open my mouth – “speaking SAS” has gone from frustrating to enjoyable. Now I can start working on getting rid of my accent, and increasing my vocabulary so that I can express myself better. In programming terms, rather than just get things done, I can look for cleaner, faster, more efficient solutions.

• I have scrolled through and inspected a dataset of ca 121,000 rows of data. Manually. Twice. Good thing there’s a Find command in the dataset viewer (no quick filter, though…)

• I’ve found myself entering a paragraph break after each sentence, and finishing each paragraph with a semicolon in an e-mail.

• I’ve started using SQL more seriously. It’s SAS’s own version of SQL, but that appears to be (a subset of) ANSI SQL with some SAS-specific additions – so hopefully most of what I learn by doing this will be useful elsewhere as well. I’ve advanced from simple “SELECT * FROM table WHERE” to queries that include joins, summary functions, groups etc. It is very powerful (which is no surprise) and some of it is a lot faster than the “pure SAS code” equivalent – and a lot more elegant.

Hmm… perhaps it’s time I got myself a SAS book. This feels like a great reason to go and do some bookshopping this weekend!

My job description doesn’t say (that is, if I actually had a job description then it wouldn’t say) a word about software development. It would probably say a fair bit about modelling, data management and data analysis, though. And once those pass a certain level in terms of complexity, you really end up in the realm of programming anyway. So while Excel is the main tool for me and my colleagues, we spend a fair amount of our time developing small applications in VBA, using VBA to augment our Excel models, and now also coding in SAS.

But we’re not programmers. And that shows.

In my case, programming has been my main hobby for the past 2 years or so, so I consider myself fairly proficient. I don’t have the level of experience that a professional developer does, but I have come to think like a developer. The other two “modellers” in our group don’t. They are very intelligent people, but when it comes to programming, they just don’t think the right way. In fact I think their general intelligence is making the problem worse, because it makes them too confident: they assume that they know what they’re doing.

Case in point:
This past Friday, I found a small but very important bug in code that colleague A had written, in a project that we are working on together. A number had been hard-coded as a constant, and in order to get the right results, that number needs to be changed manually once a month. Turns out that this number had been off by one for the last 3 months. And the output from this model is used to make million-dollar decisions.

(To make things more interesting, colleague A happened to be travelling on Friday, so I couldn’t take this up with him, but had take this directly to our manager, because he was about to use the output. The rest of the day was spent in a state of moderate panic, estimating the effects of the bug, and trying to minimise the effects.)

The bug was in no way subtle or hard to find. The report output wasn’t behaving the way I expected, and once I’d noticed that, it took less than 10 minutes to find the error.

Hard-coding a number that needs to change is dangerous and stupid in itself. But I think it’s only a symptom of a larger problem.
This really comes down to looking at code with the right mindset – and a big part of “right” is “defensive”.

1) Don’t assume that your code is correct.

The starting point should be that your code may be broken. Even if it runs, assume that it is not doing the right thing.

Every time I finish a piece of code, I ask myself – where could this go wrong? How might this output, which looks OK at first glance, be wrong? How can I check it? In the case of financial models, I usually track at least one data point in each result set all the way back to the input data, manually, using pen and paper and a calculator.

Colleague A would have found the problem if he had looked, but he didn’t – because he assumed that if it doesn’t look obviously wrong, it must be right. (It’s not the first time I find such problems in his work, either.)

It helps that I know what can go wrong. I know to look for edge cases, off-by-one errors, misaligned data, mismatched units / currencies. But these are less obvious to those who have little programming experience.

2) Failures should be visible.

Checks and double-checks and triple-checks. In environments that support it, tests and checks should be automatic. If there are two routes that should get you to the same point, take both and show both results. If there are conditions that the results should satisfy, those should be checked automatically, and very loud bells should start ringing if they don’t match.

Produce far more output than needed. Display your assumptions, and save intermediate results. Show inputs side by side with the output.

In this case, showing the key inputs would have made the problem very visible and we’d have discovered it immediately.

All of this doesn’t mean that my code is bug-free. It will never be. But it will have fewer bugs than code written by someone who doesn’t approach things this way.

Things will improve as my colleagues gain experience – but in the meantime, mistakes like this might cost us a lot of money. So I think I need to start taking more responsibility for what they do in their code.

The question is, how? How do you teach someone to think the right way – especially if they don’t even think of their job as programming?

“Leading by example” will help somewhat, but so much of this is quiet and invisible… It’s hard to make my assumptions visible. Thinking out loud while discussing our code (his and mine) might be one way. Politely adding checks and balances to the parts of his code that I review is another. But it doesn’t seem quite sufficient.

And that’s assuming that this is even “learnable” – which it might not be, it might just be a more general mindset that you either have and value, or don’t…