collapse

* Notice

Important notice (31 July): We have recently recovered from a nearly two day downtime due to an attack. No data was lost or stolen but the server has been reinstalled as a precaution. Please let us know if you encounter any issues. We apologise for the unacceptable inconvenience. Please read here for more information.

Author [EN] [PL] [ES] [PT] [IT] [DE] [FR] [NL] [TR] [SR] [AR] [RU] Topic: Python 2.7.1 for bloodlines  (Read 8852 times)

Offline ColonelAlias

  • Methuselah
  • ****
  • Posts: 327
  • Reputation: +368/-1
Python 2.7.1 for bloodlines
« on: April 15, 2011, 09:39:27 am »
Here is a project i've been doing for a while. It's a bloodlines compatible version of python 2.7.1 with almost all the standard libs (no tinker, no ssl, no sqlite - basically, nothing that required additional dependencies).  It also has the sendkeys module.

There are probably bugs remaining. I found one myself - since the C api version changed 3 times in-between python 2.1 and python 2.7, the ccmd and cvar pointers are fragile now - they are pure c pointers, and while they appear to work if used correctly (dereference something that exists), they crash if used with either non-existing attributes/functions or if they are printed with the builtin function dir.

Since this didn't work anyway in bloodlines (but didn't crash) i don't think that there is code that uses this.

It may be faster or slower, but i can't really tell on my recent computer.

While i consider this somewhat cool, i wanted to something that added more abilities for modders. I was thinking of finding this functions:
http://developer.valvesoftware.com/wiki/Developer_Console_Control#Executing_commands

To make the ccmd command functions take arguments in python files, like they cant without a tmp file workaround today.
However, the dll that contains that code appears to be obfuscated - stripped of the function names, so someone better that me at reversing things is needed to do that.


python dll and libs to run bloodlines, extract on the Bin directory
http://rapidshare.com/files/457526302/bloodlines_python27.zip
http://www.megaupload.com/?d=UDLMPTAP (mirror)
(the sh file was what i was using to test after the compile, I kept it for you to see what you need to copy and how, if you want to compile - however it is for linux (and the last line is another batch file that is missing, it just started the game in wine).


python dll source.
http://rapidshare.com/files/457526371/bloodlines_python27_source__fixed_.zip
http://www.megaupload.com/?d=0LO48TTY (mirror)

I used Visual studio 2003 to build it, the project solution is in /PC/VS7.1/pcbuild.sln.

While the code changes are actually small (three files changed), the configuration of the project was changed and the sendkeys module was added, and i can't recall exactly how, so the whole source is given.

If you want to separate only the changed parts to compile yourself from the CPython sources, check out the bottom of the file "./Python/pythonrun.c" (most code here) and the return of the DllMain function on ./PC/dl_nt.c (that changed to callback a function in pythonrun). The other file changed was a function that was moved to the pythonrun.c file - it should give a error when you try to compile with 2 definitions so you can delete the one that is in that file.



You can use this however you like, i place it on the public domain (and since i didn't use troika code except link to it, i think i can).
« Last Edit: April 15, 2011, 11:45:16 pm by ColonelAlias »

Offline ColonelAlias

  • Methuselah
  • ****
  • Posts: 327
  • Reputation: +368/-1
Re: Python 2.7.1 for bloodlines
« Reply #1 on: April 15, 2011, 10:37:04 am »
mmm the sendkeys module is not working because of a compile SNAFU. I'm fixing it and will reupload.

Edit: fixed, import SendKeys or import ctypes works again.
« Last Edit: April 15, 2011, 11:16:09 am by ColonelAlias »

Offline ColonelAlias

  • Methuselah
  • ****
  • Posts: 327
  • Reputation: +368/-1
Re: Python 2.7.1 for bloodlines
« Reply #2 on: April 15, 2011, 11:44:54 am »
Also, i didn't test this beyond the first 3 maps (haven, santamonica, beach), so if there is a horrible horrible bug, feel free to post away how this sucks.
 :razz:

Offline Lictor

  • Seductive Vampire
  • Ancillus
  • ***
  • Posts: 148
  • Reputation: +279/-3
  • Troika's Follower
    • Animoon
Re: Python 2.7.1 for bloodlines
« Reply #3 on: April 15, 2011, 01:27:14 pm »
What's better with this version ?

Offline ColonelAlias

  • Methuselah
  • ****
  • Posts: 327
  • Reputation: +368/-1
Re: Python 2.7.1 for bloodlines
« Reply #4 on: April 15, 2011, 01:53:23 pm »
You can use all python libraries (except those that had external dependencies).
Most are useless (but never loaded), but regex for instance can be useful, and other things.

The sendkeys module (and ctypes for mouse, but i didn't dive into that) is interesting too - it allows you to programmaticly "press" a keyboard key without actually pressing it.

I want to use it to simulate the dialog number option presses from the mouse, but i'm sure new uses will be found.

Finally, the python language had some features added in between 2.1 and 2.7. Generators, booleans, continuations other things i forget (never into python).

Basically: it's for modders to play with if they want.
« Last Edit: April 15, 2011, 01:59:52 pm by ColonelAlias »

Offline int9

  • Fledgling
  • *
  • Posts: 14
  • Reputation: +22/-0
Re: Python 2.7.1 for bloodlines
« Reply #5 on: April 28, 2011, 02:24:36 am »
Maybe this will help you get client commands working again.

in engine.dll:
   11 refs to "Cbuf_AddText: overflow"
   [2DCC0] is Cbuf_AddText
      38 refs to Cbuf_AddText
      [1C260] is hudClientCmd
         5 refs to hudClientCmd
            [1A570] is ClientCmd
            [108A40] is ClientCmd

ClientCmd signature:
   int __stdcall ClientCmd (const char* szCmdString);


Not sure about your other problems since I didn't understand them too well. You mentioned the ABI issues, that's an obvious problem--not sure how you worked around it--and that can be solved by introducing a translation layer which exposes legacy APIs.

Offline ColonelAlias

  • Methuselah
  • ****
  • Posts: 327
  • Reputation: +368/-1
Re: Python 2.7.1 for bloodlines
« Reply #6 on: May 01, 2011, 12:37:08 am »
I didn't work around them - except for booleans that were used in the game engine "old-style", like floats. That was easy, just translate them at the borders of the python interpreter public functions used by the engine.

It just worked after that (and setting a home relative to the executable, easy to do).
 :razz:

I did have to expose one function that was public and turned private but was not deleted in the python runtime (they said in comment that it was so linking still worked, but the function was deprecated, so i guess it's ok).


I hooked the Msg call to feed text back to the console with the GetProcAddress trick. That was easy, since that dll appears to be simple c.

However, engine dll looks like c++.
The valve faq says they are invoked like this:
engine->ClientCmd( "say hello\n" ); // execute command on client

I don't know how to hook up into the dynamic "engine" object to send those commands. That (and not being able to find them anyway) is why i gave up on that. And how can i call it anyway without the GetProcAddress trick from c?

You post seems to indicate i am mistaken about that, and the location of the functions being obfuscated. I wasn't able to find the names using ollydbg.

BTW, thanks a lot!
« Last Edit: May 01, 2011, 12:56:49 am by ColonelAlias »

Offline int9

  • Fledgling
  • *
  • Posts: 14
  • Reputation: +22/-0
Re: Python 2.7.1 for bloodlines
« Reply #7 on: May 01, 2011, 08:44:05 am »
However, engine dll looks like c++.
The valve faq says they are invoked like this:
engine->ClientCmd( "say hello\n" ); // execute command on client

I don't know how to hook up into the dynamic "engine" object to send those commands.
ClientCmd technically belongs to the IVEngineClient interface, but iirc (when I looked at the assembly code) the implementation definition used no instance variables, so it should be callable as a normal static function:
Code: [Select]
#include <cstdint>

typedef int ClientCmd_t (char const* command);

std::intptr_t engine_baseaddr = std::intptr_t(::GetModuleHandleW(L"engine.dll"));

ClientCmd_t* ClientCmd = reinterpret_cast<ClientCmd_t*>(engine_baseaddr + 0x1A570);
ClientCmd("changelevel de_dust\n");

If you want to do it the 'official' Valve way then try something like this:
Code: [Select]
struct IVEngineClient { # declared in cdll_int.h
virtual void unused1 () = 0;
virtual void unused2 () = 0;
virtual void unused3 () = 0;
virtual void unused4 () = 0;
virtual void unused5 () = 0;
virtual int __stdcall ClientCmd (char const* command) = 0;
}

typedef void* CreateInterface_t (char const* interface, int);

CreateInterface_t* CreateInterface = reinterpret_cast<CreateInterface_t*>(::GetProcAddress(
   ::GetModuleHandleW(L"engine.dll")
,  "CreateInterface"));
IVEngineClient* engine = reinterpret_cast<IVEngineClient*>(CreateInterface("VEngineClient006", 0));

engine->ClientCmd("changelevel de_dust\n");
I didn't check the vtable ordering so that declaration might be off. And since vtables are implementation-defined, this method might not be compatible with the compiler you're using. MSVC (ver6 especially) would be safest.

Apologies for the  C++ code. Hopefully it should be simple to translate to C. Dunno if it'll work though :).

That (and not being able to find them anyway) is why i gave up on that. And how can i call it anyway without the GetProcAddress trick from c?

You post seems to indicate i am mistaken about that, and the location of the functions being obfuscated. I wasn't able to find the names using ollydbg.
Nope, you weren't mistaken. The debugging information is missing because we don't have the PDBs.

HL2 SDK is useful for this. And download the pre-HL2 code leak while you're at it. Neither one will help automatically with debugging, so to identify functions like I did you need to know some asm :).

Actually, I take that back. You miiight be able to generate type libs for IDA debugging.

Offline Drathian

  • A Vampyre Modder
  • Neonate
  • **
  • Posts: 62
  • Reputation: +253/-0
    • Demon Castle Dracula
Re: Python 2.7.1 for bloodlines
« Reply #8 on: May 07, 2011, 06:15:35 am »
This is fascinating, and hello @ int9, glad you  made it here.

I wonder if the new python version supports enhanced repathing?  One problem that  always faced my clean install compilation project (int9 method)  was lack of linux support due to wine getting confused.

  Also I did have a mod here its buried somewhere I think.......

  I've been wanting to update this coffinated mod, however,  a) seems lack interest, b)   I keep getting mixed signals from wesp never ending patches lol,  and burgermeister whether the companion will be included in a future version of clan quest.   I hate waiting things out, but if thats the case there is little point in several of us working on the same thing seperately.  A completed companion + clanquest mod installer would require only a few lines of code to change for the int9 method, I like how it allows you to have cleaner mod directories and multiple mods installed at once.

 Anyways,  If linux support works it would make things very interesting indeed, I'm slowly fiddling with wine and trying to move over to Mandriva Linux full time.   So the linux/wine pathing problem was a serious blow to my efforts.

Offline ColonelAlias

  • Methuselah
  • ****
  • Posts: 327
  • Reputation: +368/-1
Re: Python 2.7.1 for bloodlines
« Reply #9 on: May 07, 2011, 06:03:23 pm »
What pathing problem?

You're writing about the need to start the game from the exe directory in wine? (the shortcuts do that in windows).

The game works on wine (with a nocd to remove the securerom shit and with the fglrx drivers at least).
« Last Edit: May 07, 2011, 06:25:08 pm by ColonelAlias »

Offline Acleacius

  • Antediluvian
  • *****
  • Posts: 927
  • Reputation: +447/-3
Re: Python 2.7.1 for bloodlines
« Reply #10 on: May 08, 2011, 06:06:23 pm »
Great work ColonelAlias and int9!   :rock:
Trust me, most of the names I have been called you can't translate in any language...they're not even real words as much as a succession of violent images.

Offline ColonelAlias

  • Methuselah
  • ****
  • Posts: 327
  • Reputation: +368/-1
Re: Python 2.7.1 for bloodlines
« Reply #11 on: May 08, 2011, 06:43:17 pm »
Truth to tell, i'm slacking. I haven't investigated yet if it's possible to do what i want with those offsets, and i don't have the HL2 beta source of IDA to try to make the lib files.

If int9 would be willing to expend more time to try to get something like that, i would love to add listeners to the vgui dialogs and show the mouse for them.

Offline Drathian

  • A Vampyre Modder
  • Neonate
  • **
  • Posts: 62
  • Reputation: +253/-0
    • Demon Castle Dracula
Re: Python 2.7.1 for bloodlines
« Reply #12 on: May 09, 2011, 07:17:24 am »
What pathing problem?

You're writing about the need to start the game from the exe directory in wine? (the shortcuts do that in windows).

The game works on wine (with a nocd to remove the securerom shit and with the fglrx drivers at least).

Just the unix placeholder "/" instead of "\", it confuses most mods in wine with the python script that int9 wrote? (not sure if its his script, the mods, or linux, or wine being the problem? dontcha love compatability layers and emulation? :taunt:)

I reference the project here:

https://forums.planetvampire.com/?topic=4875.0

Offline ColonelAlias

  • Methuselah
  • ****
  • Posts: 327
  • Reputation: +368/-1
Re: Python 2.7.1 for bloodlines
« Reply #13 on: May 09, 2011, 03:13:10 pm »
It shouldn't. Unless you're trying to use the python script outside wine (or logically, the game).

The path translation layer is one of the most tested parts of wine.

Edit: i see, you are expecting to use that outside of wine. Well, with this you could probably do it once at the start, but those kind of problems can be avoided easily.

Just like java, python can be portable. You just need to get the current script directory (which can be harder than is obvious, since the obvious way just gets you the location from where you started the script, that is not the same), and get you paths with a abstraction.


I don't know exactly how it is done in python, but if you search on google,
python getting script dir
and
python system path separator
you can do it in the obvious way.

Edit 2:
You can do it with os.path.join(sys.path[0], "MyDir")
I don't know if this has the problem of getting the current dir instead of the script dir, maybe.
« Last Edit: May 09, 2011, 03:24:29 pm by ColonelAlias »

Offline int9

  • Fledgling
  • *
  • Posts: 14
  • Reputation: +22/-0
Re: Python 2.7.1 for bloodlines
« Reply #14 on: May 09, 2011, 03:22:33 pm »
tbh I completely forgot about that mod loader attempt. I was starting to play Bloodlines, but I quit soon after that. I'd like to look into it again (preferably to add/restore a mod chooser UI) but I have some other things I want to do first. Yesterday I was working on vgui dll addons :).

iirc the problem with the mods was how relative paths were used in Python scripts--the Python interpreter always set the base dir to /vampire/python not [mod]/python. So only the base scripts were ever loaded.

 

SimplePortal 2.3.7 © 2008-2020, SimplePortal