Care with Python
Sep. 29th, 2018 04:03 pm![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png){kind=small}
mtbcFor our software at work we use Python in some components. I am not much of a fan of Python — for me it brings the worst of Perl while omitting the best — but I have been trying to become more comfortable with it. I have found it easy to write buggy code: for instance, flake8 is quite happy with,
I have also found myself having to be wary in other ways. For example, recently I tried experimenting with the multiprocessing module and if too careless then I am easily surprised. If I fire off a process to print something to standard output then return a result, then having gotten that result and written something else to standard output, those output lines may appear out of order if I do not first join() from that other process, at least unless I manually flush or adjust buffering. Or, if I pass an object as an argument to apply_async or somesuch, it may effectively get passed by value instead of by reference so its state changes are lost as side-effects: I must remember that data may be pickled on its way about. In some situations it thus turns out that I must be doubly wary.
I have a large multidimensional numpy array that I was hoping to have multiple processes read from without copying it around. On trying to work out correspondences between ctypes and dtypes and whatnot I kind of gave up there and figured that I would prefer a night's sleep.
x = 'foo'
for x in range(5):
pass
print(x)4which means that I must be very careful about tracking variable names: scoping may not be as I expect.
I have also found myself having to be wary in other ways. For example, recently I tried experimenting with the multiprocessing module and if too careless then I am easily surprised. If I fire off a process to print something to standard output then return a result, then having gotten that result and written something else to standard output, those output lines may appear out of order if I do not first join() from that other process, at least unless I manually flush or adjust buffering. Or, if I pass an object as an argument to apply_async or somesuch, it may effectively get passed by value instead of by reference so its state changes are lost as side-effects: I must remember that data may be pickled on its way about. In some situations it thus turns out that I must be doubly wary.
I have a large multidimensional numpy array that I was hoping to have multiple processes read from without copying it around. On trying to work out correspondences between ctypes and dtypes and whatnot I kind of gave up there and figured that I would prefer a night's sleep.
