Mailman 3 January 2000 - Python-Dev

Changing existing class instances
by A.M. Kuchling Feb. 9, 2000

Feb. 9, 2000

Currently, when you replace a class definition with an updated version, it's really difficult to change existing class instances; you'd have to essentially sweep every Python object and check if it's an instance, starting at roots such as __main__ and sys.modules. This makes developing code in a long-running process difficult, Zope being the best example of this. When you modify a class definition used by Zope code, you can't update existing instances floating around in memory. Over dinner, a friend and I were discussing this, and we thought it probably isn't difficult to add an extra level of indirection to allow fixing this. The only other option we could think of is either the complete scan of all objects, or inserting a forwarding pointer into PyClassObjects that points to the replacing class if !NULL, and then chase pointers when accessing PyInstanceObject->in_class. A quick hack to implement the extra indirection took about half an hour. It does these things: * Defines a PyClassHandle type: struct _PyClassHandle { PyClassHandle *next; /* ptr to next PyClassHandle in linked list */ PyClassObject *klass; /* The class object */ } ; * The in_class attribute of PyInstanceObject becomes a PyClassHandle* instead of a PyClassObject*, and all code such as inst->in_class becomes inst->in_class->klass. * As a quick hack to allow changing the class object referenced by a handle, I added a .forward( <newclassobject> ) method to class objects. This basically does self.handle->klass = <newclassobject>. The end result is that obj.__class__.forward(newclass) changes obj to be an instance of newclass, and all other instances of obj.__class__ also mutate to become newclass instances. Making this purely automatic seems hard; you'd have to catch things like 'import ftplib; ftplib.FTP = myclass', which would require automatically calling ftplib.FTP.forward( myclass ) to make all existing FTP instances mutate. Would it be worthwhile to export some hook for doing this in 1.6? The cost is adding an extra pointer deref to all access to PyInstanceObject->in_class. (This could probably also be added to ExtensionClass, and probably doesn't need to be added to core Python to help out Zope. Just a thought...) -- A.M. Kuchling http://starship.python.net/crew/amk/ Here the skull of a consumptive child becomes part of a great machine for calculating the motions of the stars. Here, a yellow bird frets within the ribcage of an unjust man. -- Welcome to Orqwith, in DOOM PATROL #22

12 39

Python 1.6 timing
by Guido van Rossum Feb. 4, 2000

Feb. 4, 2000

Andrew let me repost this mail of his to this list. It's worth a discussion here (if not in a larger forum). My responses are at the bottom. ------- Forwarded Message Date: Wed, 19 Jan 2000 20:17:55 -0500 From: "A.M. Kuchling" <amk1(a)erols.com> To: guido(a)python.org Subject: Python 1.6 timing I thought a bit more about the release schedule for 1.6, and like the idea of delaying it less and less. Another bad effect of delaying it is that not having Unicode in the core handicaps developing XML tools; we can continue working with wstrop, or integrate MAL's code into the XML-SIG's CVS tree, but it might mean abandoning the XML processing field to Perl & Tcl because the tools can't be made fully standard compliant in time. Options I can think of: 1) Delegating some control to a pumpkin holder [...]. 2) Releasing the Unicode+sre modules as separate add-ons to 1.5. (But would that impose annoying backward-compatibility constraints when they get integrated into 1.6?) 3) Add Unicode, sre, Distutils, plus other minor things and call it 1.5.5, meaning it's not as big a revision as a 1.6 release, but it's bigger than just another patchlevel of bugfixes. I don't remember what other features were planned for 1.6; was there anything major, if static typing is left for 2.0? - -- A.M. Kuchling http://starship.python.net/crew/amk/ Life's too short for chess. -- H.J. Byron ------- End of Forwarded Message There are several other things I can think of now that were planned for 1.6: revamped import, rich comparisons, revised coercions, parallel for loop (for i in L; j in M: ...), extended slicing for all sequences. I've also been thinking about making classes be types (not as huge a change as you think, if you don't allow subclassing built-in types), and adding a built-in array type suitable for use by NumPy. I've also received a conservative GC patch that seems to be fairly easy to apply and has some of Tim Peters' blessing. For 1.5.5 (what happened to 1.5.3 and 1.5.4?), we can have a more conservative agenda, as suggested by Andrew: Unicode and distutils are probably the most important things to integrate. (The import utilities are not ready for prime time in my opinion; there are too many issues.) Anybody care to be the pumpkin? That would cut the discussion short; otherwise the problem remains that I can't spend too much time on the next release unless I get funded for it; what little money I've received for CP4E I had better spend on getting some CP4E-related results ASAP, because the next installment of this funding is very much at stake... --Guido van Rossum (home page: http://www.python.org/~guido/) Life's better without braces. -- Bruce Eckel

20 70

Adding C ternary select (a?b:c) to Python?
by Guido van Rossum Feb. 2, 2000

Feb. 2, 2000

Dear developers, Eric Raymond has sent me the following patch, which adds conditional expressions to Python. I'd like to hear opinions on whether this is a good thing to add to Python, and whether this is the right syntax. I am a bit skeptical about whether this is sufficiently Pythonic, but on the other hand there have always been requests for such a feature, and the existing solutions are ugly: a and b or c only works when you know for sure that b will never be false, and (a and [b] or [c])[0] is dead ugly... --Guido van Rossum (home page: http://www.python.org/~guido/) Subject: Ternary select -- here it is. From: "Eric S. Raymond" <esr(a)thyrsus.com> To: Guido Van Rossum <guido(a)CNRI.Reston.Va.US> Date: Sat, 29 Jan 2000 17:40:31 -0500 X-Eric-Conspiracy: There is no conspiracy Dang, Guido, this was *fun*! This patch extends the Python interpreter to support the C ternary operator, and documents the extension in the Reference Manual. The implementation is dead simple and robust: it's adapted from the existing code for if statements, and adds or changes less than 70 lines of code all told. Diffs between last version checked in and current workfile(s): --- Grammar/Grammar 2000/01/28 17:10:18 1.1 +++ Grammar/Grammar 2000/01/29 22:14:05 @@ -61,7 +61,8 @@ except_clause: 'except' [test [',' test]] suite: simple_stmt | NEWLINE INDENT stmt+ DEDENT -test: and_test ('or' and_test)* | lambdef +test: bool_test ['?' bool_test ':' bool_test] +bool_test: and_test ('or' and_test)* | lambdef and_test: not_test ('and' not_test)* not_test: 'not' not_test | comparison comparison: expr (comp_op expr)* --- Include/token.h 2000/01/28 17:38:55 1.1 +++ Include/token.h 2000/01/29 01:27:00 @@ -74,10 +74,11 @@ #define LEFTSHIFT 34 #define RIGHTSHIFT 35 #define DOUBLESTAR 36 +#define QUERY 37 /* Don't forget to update the table _PyParser_TokenNames in tokenizer.c! */ -#define OP 37 -#define ERRORTOKEN 38 -#define N_TOKENS 39 +#define OP 38 +#define ERRORTOKEN 39 +#define N_TOKENS 34 /* Special definitions for cooperation with parser */ --- Modules/parsermodule.c 2000/01/28 18:03:27 1.1 +++ Modules/parsermodule.c 2000/01/29 22:13:45 @@ -945,6 +945,7 @@ #define validate_star(ch) validate_terminal(ch, STAR, "*") #define validate_vbar(ch) validate_terminal(ch, VBAR, "|") #define validate_doublestar(ch) validate_terminal(ch, DOUBLESTAR, "**") +#define validate_query(ch) validate_terminal(ch, QUERY, "?") #define validate_dot(ch) validate_terminal(ch, DOT, ".") #define validate_name(ch, str) validate_terminal(ch, NAME, str) @@ -963,7 +964,8 @@ VALIDATER(exec_stmt); VALIDATER(compound_stmt); VALIDATER(while); VALIDATER(for); VALIDATER(try); VALIDATER(except_clause); -VALIDATER(test); VALIDATER(and_test); +VALIDATER(test); +VALIDATER(bool_test); VALIDATER(and_test); VALIDATER(not_test); VALIDATER(comparison); VALIDATER(comp_op); VALIDATER(expr); VALIDATER(xor_expr); VALIDATER(and_expr); @@ -1829,12 +1831,34 @@ } /* validate_except_clause() */ +/* bool_test ( | bool_test ? bool_test ) + * + */ static int validate_test(tree) node *tree; { + if (!validate_ntype(tree, test)) + return 0; + else if (NCH(tree) == 1) + return(validate_bool_test(CHILD(tree, 0))); + else if (validate_numnodes(tree, 5, "expr")) + { + return validate_bool_test(CHILD(tree, 0)) + && validate_query(CHILD(tree, 1)) + && validate_bool_test(CHILD(tree, 2)) + && validate_colon(CHILD(tree, 3)) + && validate_bool_test(CHILD(tree, 4)); + } +} /* validate_test() */ + + +static int +validate_bool_test(tree) + node *tree; +{ int nch = NCH(tree); - int res = validate_ntype(tree, test) && is_odd(nch); + int res = validate_ntype(tree, bool_test) && is_odd(nch); if (res && (TYPE(CHILD(tree, 0)) == lambdef)) res = ((nch == 1) @@ -1848,7 +1872,7 @@ } return (res); -} /* validate_test() */ +} /* validate_bool_test() */ static int --- Parser/tokenizer.c 2000/01/28 17:37:48 1.1 +++ Parser/tokenizer.c 2000/01/29 01:27:26 @@ -99,6 +99,7 @@ "LEFTSHIFT", "RIGHTSHIFT", "DOUBLESTAR", + "QUERY", /* This table must match the #defines in token.h! */ "OP", "<ERRORTOKEN>", @@ -384,6 +385,7 @@ case '}': return RBRACE; case '^': return CIRCUMFLEX; case '~': return TILDE; + case '?': return QUERY; default: return OP; } } --- Python/compile.c 2000/01/28 23:17:19 1.1 +++ Python/compile.c 2000/01/29 22:19:29 @@ -1698,11 +1698,11 @@ } static void -com_test(c, n) +com_bool_test(c, n) struct compiling *c; node *n; { - REQ(n, test); /* and_test ('or' and_test)* | lambdef */ + REQ(n, bool_test); /* and_test ('or' and_test)* | lambdef */ if (NCH(n) == 1 && TYPE(CHILD(n, 0)) == lambdef) { PyObject *v; int i; @@ -1738,6 +1738,32 @@ } static void +com_test(c, n) + struct compiling *c; + node *n; +{ + int op; + REQ(n, test); + com_bool_test(c, CHILD(n, 0)); + + /* is there a following ternary operator? */ + /* XXX optimize the compilation when the guard is a constant */ + if (NCH(n) == 5) + { + int anchor1 = 0, anchor2 = 0; + com_addfwref(c, JUMP_IF_FALSE, &anchor2); + com_addbyte(c, POP_TOP); + com_pop(c, 1); + com_node(c, CHILD(n, 2)); + com_addfwref(c, JUMP_FORWARD, &anchor1); + com_backpatch(c, anchor2); + com_addbyte(c, POP_TOP); + com_node(c, CHILD(n, 4)); + com_backpatch(c, anchor1); + } +} + +static void com_list(c, n, toplevel) struct compiling *c; node *n; @@ -2931,6 +2957,9 @@ break; case test: com_test(c, n); + break; + case bool_test: + com_bool_test(c, n); break; case and_test: com_and_test(c, n); --- Doc/ref/ref5.tex 2000/01/29 21:28:13 1.1 +++ Doc/ref/ref5.tex 2000/01/29 22:00:02 @@ -764,7 +764,7 @@ \section{Boolean operations\label{Booleans}} \indexii{Boolean}{operation} -Boolean operations have the lowest priority of all Python operations: +Boolean operations have the lowest priority of all Python binary operations: \begin{verbatim} expression: or_test | lambda_form @@ -832,6 +832,24 @@ def make_incrementor(increment): return lambda x, n=increment: x+n \end{verbatim} + +\section{Select\label{select}} +\index{select} + +The select operator is a ternary operator with lower priority than +boolean operations (and thus lower priority than all other binary and +unary operators). + +\begin{verbatim} +select_expr: xor_expr | xor_expr "?" xor_expr ":" xor_expr +\end{verbatim} + +If its first operand is nonempty, the value of a select operation is +its second operand; otherwise the value is the third operand. + +(The semantics and precedence level of select are intended to be +unsurprising to programmers familiar with \C's ternary select +operator.) \section{Expression lists\label{exprlists}} \indexii{expression}{list} End of diffs. -- <a href="http://www.tuxedo.org/~esr">Eric S. Raymond</a> You know why there's a Second Amendment? In case the government fails to follow the first one. -- Rush Limbaugh, in a moment of unaccustomed profundity 17 Aug 1993

20 46

Ann: Stackless Python 1.02
by Christian Tismer Jan. 31, 2000

Jan. 31, 2000

Stackless Python is a Python without C stack usage. It allows for all kinds of non-local control flow. For info see http://www.tismer.com/research/stackless/ Update for Stackless Python V. 1.02: - passes all standard tests - should work with Zope now (try...finally was incorrect) - has a smart object destructor for really deeply nested stuff The "5 percent speedup" is no longer there currently, since the smart object destructor needs to be optimized into the objects. It is not done in this test phase but will come. Please visit the Stackless Python homepage at http://www.tismer.com/research/stackless/ Fact sheet, links to documentation, source and binaries can be found there. cheers - chris <P><A HREF="http://www.tismer.com/research/stackless/">Stackless Python 1.02 + Continuations 0.6</A> - a version of Python 1.5.2 that does not need space on the C stack, and first-class callable continuation objects for Python. (31-Jan-2000) Christian Tismer Mission Impossible 5oftware Team

1 0

Riskless deletion of nested structures
by Christian Tismer Jan. 31, 2000

Jan. 31, 2000

Howdy, Please review! While implementing Stackless Python, a new problem arose: Nested structures like frame chains and tracebacks can now easily grow somuch that they cause a stack overflow on deallocation. To protect lists, tuples, frames, dicts and tracebacks against this, I wrote a stackless deallocator. At the moment, everything is done in trashcan.c . This gives a slight performance loss of 5% for pystone, most probably due to the double indirection and non-local code reference. It is yet a hack, since I'm grabbing the tp->dealloc pointers of these types and replace them by safe versions. This just in order to try out things quickly. Later I will change this and incorporate the stack checks into the affected modules, after I got some feedback on this. This patch applies to Stackless and standard Python as well: Deallocation of deeply nested structures will never again cause a stack overflow. Installation for the intermediate version: Insert a line _Py_trashcan_install(); at the end of Py_Initialize in pythonrun.c Please try it and check my code wether there is a better solution. cheers - chris -- Christian Tismer :^) <mailto:tismer@appliedbiometrics.com> Applied Biometrics GmbH : Have a break! Take a ride on Python's Düppelstr. 31 : *Starship* http://starship.python.net 12163 Berlin : PGP key -> http://wwwkeys.pgp.net PGP Fingerprint E182 71C7 1A9D 66E9 9D15 D3CC D4D7 93E2 1FAE F6DF we're tired of banana software - shipped green, ripens at home

1 0

Eight suggestions for Python books
by gvwilson＠nevex.com Jan. 30, 2000

Jan. 30, 2000

I realize this is a bit off-topic, but based on discussions with various publishers over the last couple of years, and on conversations at the conference last week, I wrote up some notes on books that I think would help Python succeed, and posted them to comp.lang.python under the heading "Eight suggestions for Python books". Some of the books are specificially about Python, while others try to use Python to fill holes in the curriculum, and thereby get Python into colleges through the side door (in the same way that networking and computer graphics texts helped C become more popular). Hope it's useful... Greg

1 0

Multiple dicts for string interpolation?
by Skip Montanaro Jan. 28, 2000

Jan. 28, 2000

Every once in awhile I want to perform string interpolation using more than one dictionary. One way is to build a dictionary that's a union of multiple dictionaries: dict = {} dict.update(d1) dict.update(d2) ... s = format % dict Another way is the MultiDict approach that Digital Creations (used to?) use in their DocumentTemplate module (I can't remember the exact usage any more): dict = MultiDict() dict.append(d1) dict.append(d2) ... s = format % dict A MultiDict object maintains a list of the dicts it's been fed and searches them in order when __getitem__ is called. I'd like to propose a third alternative. How about if the string interpolation function accepted a tuple of dictionaries directly: s = format % (d1, d2) It would only be used when named interpolation was expected. I don't think there would be any conflict with current % operator semantics. Skip Montanaro | http://www.mojam.com/ skip(a)mojam.com | http://www.musi-cal.com/ 847-971-7098

6 11

To-do for Distutils 0.2 (and beyond)
by Greg Ward Jan. 28, 2000

Jan. 28, 2000

Hi all -- [oops: I *said* that I was cc'ing this to python-dev when I posted it to distutils-sig, but I just plain forgot to. So here it is again, still presumed relevant to python-dev'ers because it directly affects the planned Python 1.6 feature set. Please reply either to me directly or to distutils-sig, because that's where the implementation will be done.] recent developments in the planned release schedule for Python 1.6 (Guido doesn't want to wait for everything planned, but get something out the door in the next couple of months) are lighting a fire under me to get a seriously usable version of Distutils ready *really* soon now. This will be Distutils 0.2; I anticipate that 0.2.x will be included in Python 1.6, where x is the number of attempts it takes me to get something reasonably bug-free out the door. Those if you who were at the Python Conference this past week will have seen bits and pieces of my "laundry list" of desired features that should be added to the Distutils at some point in the future. Given the shortened schedule, it looks like it's necessary to do some pruning and concentrate on the essentials to get something in Python 1.6. So, here is my current laundry list, sorted into a couple of categories: essential for 0.2 (Python 1.6) ----------------- * fix the "dist" command (two-phase manifest, better feedback) (steal ideas and hopefully code from Gordon Macmillan's installer) * fix the "install" command (have the right interface and do the right thing for installating to non-standard or personal directories) * fix some bad nomenclature in the command classes (most likely 'options' -> 'user_options', 'set_default_options()' -> 'initialize_options()', and 'set_final_options()' -> ??? (maybe 'finalize_options()'?)) * build C libraries (for PIL, and other similar distributions) * documentation (two small manuals that should become standard Python manuals: "Installing Python Modules" and "Developing and Distributing Python Modules") * add a bdist command; should at least be able to generate dumb archives of built distributions (eg. a tarball that you unpack from /usr/local, or maybe /usr/local/lib/python1.5/site-packages) desirable for 0.2 ----------------- * "what's installed" database * dependency checking ("is version 1.3 of foo installed?") * don't automatically clobber an existing installation -- confirm, or require a "force" option, or something * command to install C headers (assuming more extensions than NumPy need to do this) put off to 0.3 (Python 1.7?) ---------------------------- * JPython support (most importantly, know where to install .py modules when run from JPython and be able to build Java extensions for JPython) * build static Python binary (for shared-library-challenged OSs) * SWIG support (mainly, know how to run it before building the C extension it generates) * PyFort support (ditto) * Mac support * allow overlapping multi-architecture installations (Michel Sanner's pet peeve and Guido's nightmare ;-) (this would *not* be the default; it would have to be explicitly chosen by brave/cheap/foolhardy installers) * support for archive sites (Parnassus) to pull out meta-info Anyone feel strongly about moving any of these items around, or discarding any entirely, or adding anything? Please let me know by email; I'll summarize to the list. Thanks -- Greg

2 1

Re: [Python-Dev] Multiple dicts for string interpolation?
by Jack Jansen Jan. 26, 2000

Jan. 26, 2000

> the part you didn't like about that? If so, how about changing the > constructor to > > def __init__(self, *dicts): > ... > > instead so you could use it as a one-liner > > format % MultiDict(d1, d2, ...) > > ? That's exactly the same as the tuple idea, except there's a nice > descriptive word in the middle of it <wink>. I've always wonderer why dict+dict isn't supported (or possibly dict|dict, if the key-collision semantics of + on dict are seen as a problem). Is there a good reason for this, or is it just that there are other more important things to implement? This wouldn't be a replacement for all uses of MultiDict, as it would probably have to create a new dict to keep semantics in line with those of list+list -- Jack Jansen | ++++ stop the execution of Mumia Abu-Jamal ++++ Jack.Jansen(a)oratrix.com | ++++ if you agree copy these lines to your sig ++++ www.oratrix.nl/~jack | see http://www.xs4all.nl/~tank/spg-l/sigaction.htm

2 1

Re: [Python-Dev] Python 1.6 timing (fwd)
by gvwilson＠nevex.com Jan. 22, 2000

Jan. 22, 2000

Guido> There are several other things I can think of now that were Guido> planned for 1.6: revamped import, rich comparisons, revised Guido> coercions, parallel for loop (for i in L; j in M: ...), Guido> extended slicing for all sequences. I've also been thinking Guido> about making classes be types (not as huge a change as you Guido> think, if you don't allow subclassing built-in types), and Guido> adding a built-in array type suitable for use by NumPy. I've Guido> also received a conservative GC patch that seems to be fairly Guido> easy to apply and has some of Tim Peters' blessing. BAW> All very cool things that could easily wait until 1.7. After BAW> all, what's in a number? GVW writes: I agree on all counts except garbage collection --- I'm half-way through the second day of the Python class I teach at Los Alamos (the people who are funding the Python tool design competition), and it's come up a couple of times. People want to be able to prototype meshes, throw callbacks around without worrying about circularity, and some other things that I don't really understand yet. There's also a couple of smart guys in the class who are wondering about CPython vs. JPython ("So this'll be safe in one version of the language, but not in the other?"), and about marketing ("Help me win a feature comparison against Java in my group..."). There's also been questions about tighter integration of NumPy (e.g. overloading operators rather than calling 'greater()' to do comparison), but I think that's a separate discussion... My $0.02, Greg

5 8