Comments on: WYS is not always WYG in python.re

By: Serge

Serge — Sun, 05 Oct 2008 01:45:54 +0000

‘总数’.decode(‘utf-8′)

is equal to

# -*- coding: utf-8 -*-
u’总数’

Notice the u prefix

By: Mikko Ohtamaa

Mikko Ohtamaa — Sat, 04 Oct 2008 14:01:42 +0000

I do not posses skills in Python regular expression madness, but I suggest you to try to use BeautifulSoup module for HTML pattern matching.

By: Bob Ippolito

Bob Ippolito — Fri, 03 Oct 2008 16:16:31 +0000

I think the problem is that you’re not actually using unicode, you’re still using strings.

Instead of this:
pattern = re.compile(‘(?<=总数 )(?P\d+)’, re.UNICODE)

You should do this:
pattern = re.compile(u’(?<=总数 )(?P\d+)’, re.UNICODE)

Notice the “u” prefix on the string literal. That makes it a unicode literal.

By: zakhar

zakhar — Fri, 03 Oct 2008 14:22:44 +0000

u’(?<=总数 )(?P\\d+)’ == ‘(?<=总数 )(?P\d+)’.decode(‘utf-8′)

By: zakhar

zakhar — Fri, 03 Oct 2008 14:17:30 +0000

I think this code must work:
pattern = re.compile(u‘(?<=总数 )(?P\\d+)’, re.UNICODE)

By: Thomas Waldmann

Thomas Waldmann — Fri, 03 Oct 2008 10:31:31 +0000

You’re doing it too complicated.

# -*- coding: utf-8 -*-
content = urllib.urlopen(url).read()
content = content.decode(coding)

pattern = re.compile(ur’总数 (?P\d+)’, re.UNICODE)

Please note:
* strictly taken, you should get the page content encoding from the page
* i don’t think you need that look-before assertion
* but what you definitely need is u’…’ or ur’…’!