Friday, August 13, 2010
PyPy GSoC: Wrapping up
Well, we're reaching the end of the official GSoC period, and asmgcc-64 still has some obscure bugs. I'm going to continue to work on it in the two weeks before my college starts up, but it's hard to make time estimates regarding how long it will take to track down weird failure modes.
But the bottom line is that the main goal of my GSoC was accomplished: A working 64-bit PyPy JIT. Hopefully I'll be able to complete asmgcc-64, and make the JIT even faster.
Monday, August 02, 2010
PyPy GSoC: 64-bit JIT merged to trunk
Well, that took longer than it should have. In any case, 64-bit JIT support (with Boehm GC only) has been merged to trunk. Please try it out!
The remainder of my GSoC will be work on the asmgcc-64 branch, to allow the 64-bit JIT to work with non-Boehm collectors.
Monday, July 26, 2010
PyPy GSoC: Nearing the merge to trunk
After some discussion with the core PyPy developers, it's been decided that the 64-bit backend will be merged as soon as possible, with only Boehm GC support. After the merge, I will continue work on asmgcc-64 to provide further speed improvements.
The x86-64-jit-backend branch is keep fairly close to trunk, so I thought the merge would be fairly trivial. Unfortunately, I spent the morning tracking down what turned out to be simple bug in the tests themselves, and the afternoon investigating a regression involving the 32-bit hybrid GC support, which I am still working on. However, I am confident that I will be able fix the regression and complete the merge this week. (knocks on wood)
Monday, July 19, 2010
PyPy GSoC: asmgcc update
Apologizes for my lapse in updates. And thanks the people who bugged me on IRC for a status update. :)
I spent some time familiarizing myself with the purpose and implementation of trackgcroot.py, then started working on porting it to support the 64-bit instruction set, which has been fairly straightforward.
Currently I am using the asmgcc test suite to iterate through the "run tests -> investigate failure -> fix bug -> repeat" cycle. Once I have 64-bit asmgcc working, I want to eliminate some code duplication that has crept into my codebase. I expect to be working on asmgcc-64 for the rest of this week at the least.
Friday, July 02, 2010
PyPy GSoC: 64-bit JIT working, but GC is slow
Finally, I have a working 64-bit JIT in my branch, although only with the slow Boehm garbage collector.
Next week, I will start work on adding 64-bit support to the asmgcc root finding strategy in order to make use of one of PyPy's faster, preferred garbage collectors.
Friday, June 25, 2010
PyPy GSoC status update
Well, the backend is RPythonic again. And I managed to remove some of the boilerplate/duplicated code from the rx86 module, although there is still room for improvement. Also, some common cases of loading a large number of variables from memory locations that are "close" to one another became more efficient.
The main thing this coming week is to address an issue that I only became fully aware of in the middle of this week: Many places in the code use jumps with a 32-bit relative displacement that are sometimes patched later to point to a different destination. The problem is sometimes you need to patch it to point to a location farther away.
This issue had been masked by the fact that on some operating systems (Linux included), you can give the operating system a hint as to where you'd like the memory to be allocated, and as a result, all of the memory used by the JIT was within 32 bits of each other. But we can't count on that working all the time on all platforms, so we need to handle the edge case.
We have a plan for how to fix this while not making the common case any less efficient, and while it is not hugely difficult, it also isn't trivial.
After this issue is taken care of, the next thing will be to port the asmgcc root-finding strategy to 64-bit. (Without this, 64-bit JIT would have to use the very slow boehm garbage collector, which sort of defeats the purpose of a JIT)
Friday, June 11, 2010
PyPy GSoC update: Most tests pass on 64-bit and 32-bit CPUs
Most JIT tests now pass on both 32-bit and 64-bit CPUs. However, there are some caveats:
- I made some changes that make the code no longer "RPythonic", which means you can't actually build a working, JIT-enabled, PyPy with this code quite yet.
- The rx86 module has accumulated some code duplication.
- There are a number of miscellaneous hacks I had to add that I'd like to get rid of somehow.
- 64-bit memory references aren't as efficient (in terms of code size) as they could be.
My plan for next week is to spend some time cleaning up the rx86 module, and then working on making the JIT translatable again. If by some miracle I get both of those things done next week, I'll start on the other two items. :-)
Monday, June 07, 2010
rx86 conversion complete, 64-bit work started
Last week saw the completion of the rx86 conversion: All tests pass, and the JIT works on i386 machines.
Last week I started on actually implementing 64-bit support. Mostly this will be an exercise in "magic removal": Locating those parts of the code that are only applicable to i386, and either adapting them to work with both i386 and x86_64 or else factoring those differences out into arch-specific classes.
One issue specific to x86_64 is the fact that most instructions cannot encode a 64-bit immediate value. I've worked around this by setting aside one register as a scratch register, and loading the immediate to that register first when a case like this arises. In the future I may try to optimize this to output a 32-bit relative displacement in the cases where that could work.
Monday, May 31, 2010
PyPy GSoC update: rx86 encoding almost done
I've almost reached the first milestone. Almost all tests pass, and I've successfully translated pypy-c-jit and run a few benchmarks. Unless I hit an unforeseen issue, I'll be able to actually start on 64-bit support sometime this week.
Friday, May 21, 2010
GSoC status update: End of week 1
GSoC doesn't officially start until May 24th, but my classes were over, so I spend this week getting started on my project. I spent the week trying to port the backend to use the new rx86 encoding scheme, as well as extended rx86 as needed. Progress was slowed quite a bit by my unfamiliarity with PyPy; it seemed every couple minutes I had to stop and spend some time just reading code, trying to figure out what it was even supposed to do. However, I was definitely getting more done by the end of the week.
By the end of next week, I hope to be finished porting the backend to rx86. After that, I'll begin actually adding 64-bit support.
