BAZOMBO: a discursive miscellany of topics

Rekindling old affection

Friday, May 09 2008

Yesterday a friend pointed me at Burger Fuel’s website. I can’t go into why, but anyway I idly poked around, as you do, and noticed something wonderful on this page: correct use of the DL, DT and DD tags.

I had forgotten all about these old friends, which together make up the definition list flavour of HTML lists.

Definition List

Definition lists, created using the DL element, generally consist of a series of term/definition pairs.

They are so obscure that the editor I use to compose these posts doesn’t even have buttons for them. I am just itching now for something to come my way that will justify my using DL and friends once more. Somehow they take me back to the early days when Mosaic was all there was and we hadn’t realised that presentational markup would lead to BLINK , MARQUEE and other atrocities.

no comments

Tags: nostalgia ~ html ~ markup

Sanitising smelly text

Tuesday, December 04 2007

At work we are migrating an old site to a new CMS.

Unfortunately the content is a mess. Owing to people pasting text in from Word and various other accidents, one fragment of HTML can be a mixture of UTF-8 and Latin-1 and cp1252 and goodness knows what else. When you’ve been a good boy and coded all your templates to declare “I am UTF-8, honest guv” it’s a bit trying. Especially when the client complains.

The markup is pretty broken too. It’s littered with weird markup from Word and generally non-compliant.

So far I’m having good results from a pipeline of various tricks.

1. Python’s unicode function. unicode takes a string and transcodes it into Unicode. You can optionally force it to treat input as a particular encoding, and you can tell it how to handle errors.
2. Beautiful Soup. It finds tag soup delicious. It also does a best-effort to detect encodings and transcode to Unicode. (You have to love software with a module called UnicodeDammit).
3. htmltidy, in its utidylib manifestation. Does beautiful cleanup. It’s not super-robust though; I can make it segfault and dump core by feeding it the crap we have. Which is why I clean up with BeautifulSoup first.
4. I butchered Josh Goldfoot’s marvellous XSS-defense script to strip out some of the more outrageous markup that I know we won’t use.

The only downside is that over thousands of items, this is pretty slow. But it’s the price you pay to be beautiful, I guess.

no comments

Tags: python ~ unicode ~ markup ~ programming ~ html tidy ~ beautiful soup

---