Prior to v1.2, in order to deodex an odex file, baksmali required a helper binary named deodexerant than ran on the phone, and provided information to baksmali about the classes loaded by dalvik, dumps of the virtual method table for classes, field offsets, inline methods, superclasses, etc. The previous deodex instructions for deodexerant can be found at http://code.google.com/p/smali/wiki/DeodexInstructions_Deodexerant"
General information about odex files
In short, an odex file is an optimized version of a classes.dex file that has optimizations that are device specific. In particular, an odex file has dependencies on every "BOOTCLASSPATH" file that is loaded when it is generated. The odex file is only valid when used with these exact BOOTCLASSPATH files. dalvik enforces this by storing a checksum of each file that the odex file is dependent on, and ensuring that the checksum for each file matches when the odex file is loaded.
The BOOTCLASSPATH is simply a list of the jars/apk from which classes can be loaded, in addition to the main apk/jar that is loaded. A normal android system has 5 jars in it's base BOOTCLASSPATH - core.jar, ext.jar, framework.jar, android.policy.jar and services.jar. These can all be found in /system/framework. However, some apks have dependencies on additional jar or apks files beyond that of the base 5 jars. For example, for applications that use google maps, com.google.android.maps.jar will be appended to the BOOTCLASSPATH for that application's apk.
These odex dependencies make life a bit difficult for a couple of reasons. For one - you can't take an apk+odex file from one system image and run it on another system image (unless the other system image uses the exact same framework files). Another problem is that if you make any changes to any of the BOOTCLASSPATH files, it will invalidate every odex that depends on that file - basically every apk/jar on the device.
The examples below assume you are using the baksmali wrapper script available on the downloads page to call baksmali. If you are calling the jar directly, you can replace "baksmali" with "java -jar baksmali.jar" instead
In order for baksmali to be able to deodex something, it has to load every BOOTCLASSPATH file that the odex depends on. By default, baksmali will try to find the 5 "core" BOOTCLASSPATH files in the current directory. It can use either a jar/apk (with a classes.dex inside), or an odex file (in which case the corresponding jar/apk isn't needed).
You can simply use the -x option to tell baksmali that it should deodex the input. If the input isn't an odex file, the option will be ignored. For example:
baksmali -x Calculator.odex
You can specify the BOOTCLASSPATH files to use with the -c option. This option has two modes of operation. If the option value does not start with a colon ':', then it is used as the entire BOOTCLASSPATH. Alternatively, if the value does start with a colon ':', then any entries are appended to the default BOOTCLASSPATH instead. In either case, multiple entries are separated with a colon ':'. For example, assuming "blah.odex" is an odex file that additionally depends on com.google.android.gtalkservice.jar:
baksmali -c :com.google.android.gtalkservice.jar -x blah.odex
which is equivalent to
baksmali -c core.jar:ext.jar:framework.jar:android.policy.jar:services.jar:com.google.android.gtalkservice.jar -x blah.odex
In either case, it will look for the 5 base BOOTCLASSPATH files along with gtalkservice in the current directory.
If one or more of the BOOTCLASSPATH files aren't in the current directory, you can add a directory to search with the -d option. You can't add multiple directories with a single -d option, but you can specify multiple -d options. For example, if you pulled the /system partition off of a phone, and you are currently in the system/app folder, then you would specify -d ../framework, to have it look in the correct place for the BOOTCLASSPATH files.
baksmali -d ../framework.jar -x Calculator.odex
When you are deodexing files, there are a couple of common issues you may run into.
Missing BOOTCLASSPATH files
The most common issue, which is almost guaranteed to happen if you're deodexing an entire build, is an error which is something like "org.jf.dexlib.Code.Analysis.ValidationException: class Lcom/google/android/gtalkservice/IChatListener; cannot be resolved."
This is caused when the odex file has an additional dependency, beyond the jars in the normal BOOTCLASSPATH. To resolve the issue, you need to find which jar contains the class mentioned in the error message, and then add that to the BOOTCLASSPATH with the -c option. You can usually guess which jar it is in from the class name, but if not, you can disassemble the framework jars and find which one has that class.
No classes.dex file
There is currently an issue in baksmali so that if it finds a BOOTCLASSPATH jar file without a classes.dex file, it will immediately bomb out, instead of continuing to look (i.e. for an odex file). If you run into this, the error message will be something like "org.jf.dexlib.Util.ExceptionWithContext: zip file core.jar does not contain a classes.dex file".
A temporary work around is to delete or rename the BOOTCLASSPATH jar files, leaving only the odex files.
Another issue you may run into is when it runs out of heap memory. The error might look something like "java.lang.OutOfMemoryError: Java heap space". To resolve this, you can use the -Xmx parameter to increase the heap size. A reasonable size to try would be 512m.You can increase it further if needed, of course. If you are using the wrapper script to call baksmali, you can use -JXmx instead. For example
baksmali -JXmx512m -x blah.odex
Or if you are calling the jar directly with something like "java -jar baksmali.jar":
java -Xmx512m -jar baksmali.jar -x blah.odex'via Blog this'