1
0
mirror of https://github.com/elua/elua.git synced 2025-01-08 20:56:17 +08:00
elua/mkfs.py

131 lines
3.9 KiB
Python
Raw Normal View History

2008-07-29 11:08:54 +00:00
# A script to convert an entire directory to a C array, in the "romfs" format
import os, sys
import re
import struct
_crtline = ' '
_numdata = 0
_bytecnt = 0
maxlen = 30
2008-07-29 11:08:54 +00:00
# Line output function
def _add_data( data, outfile, moredata = True ):
global _crtline, _numdata, _bytecnt
_bytecnt = _bytecnt + 1
if moredata:
_crtline = _crtline + "0x%02X, " % data
else:
_crtline = _crtline + "0x%02X" % data
_numdata = _numdata + 1
if _numdata == 16 or not moredata:
outfile.write( _crtline + '\n' )
_crtline = ' '
_numdata = 0
# dirname - the directory where the files are located.
2008-07-29 11:08:54 +00:00
# outname - the name of the C output
# flist - list of files
EXPERIMENTAL FEATURE: romfs compression/precompilation. Added another "romfs" parameter to SConstruct. It can take one of 3 values: - "verbatim" (default): copy all the files from the eLua romfs/ directory to the binary image (exactly what happened until now) - "compress": use LuaSrcDiet (http://luaforge.net/projects/luasrcdiet, now included in the eLua distribution) to "compress" the source code by using different tricks (shorter identifiers, removing comments and EOLS, and so on). The output is still a compilable Lua file (although in most case it looks completely different from the original) which is copied in the eLua binary image instead of the original Lua file. The compression obtained by using this method seems to be very good in practice, thus it will save flash when needed. - "compile": use the eLua cross compiler (that must be built first by running 'scons -f cross-lua.py') to precompile the Lua files to bytecode, and write the bytecode to the image instead of the source file. This way, one can save both time (the scripts don't need to be compiled anymore) and RAM (for the same reason, especially since the Lua parser uses the stack a lot, which can lead to very nasty and hard to diagnose stack overflow bugs). It will be even more useful in the future, when (hopefully) more and more Lua bytecode data structures will be available directly from Flash (without having to copy them RAM as it happens now). It might also decrease the romfs memory footprint, but then again, it might not; it pretty much depends on the Lua programs included in the romfs.
2009-12-02 19:33:03 +00:00
# mode - preprocess the file system:
# "verbatim" - copy the files directly to the FS as they are
# "compile" - precompile all files to Lua bytecode and then copy them
# "compress" - keep the source code, but compress it with LuaSrcDiet
# compcmd - the command to use for compiling if "mode" is "compile"
2008-07-29 11:08:54 +00:00
# Returns True for OK, False for error
EXPERIMENTAL FEATURE: romfs compression/precompilation. Added another "romfs" parameter to SConstruct. It can take one of 3 values: - "verbatim" (default): copy all the files from the eLua romfs/ directory to the binary image (exactly what happened until now) - "compress": use LuaSrcDiet (http://luaforge.net/projects/luasrcdiet, now included in the eLua distribution) to "compress" the source code by using different tricks (shorter identifiers, removing comments and EOLS, and so on). The output is still a compilable Lua file (although in most case it looks completely different from the original) which is copied in the eLua binary image instead of the original Lua file. The compression obtained by using this method seems to be very good in practice, thus it will save flash when needed. - "compile": use the eLua cross compiler (that must be built first by running 'scons -f cross-lua.py') to precompile the Lua files to bytecode, and write the bytecode to the image instead of the source file. This way, one can save both time (the scripts don't need to be compiled anymore) and RAM (for the same reason, especially since the Lua parser uses the stack a lot, which can lead to very nasty and hard to diagnose stack overflow bugs). It will be even more useful in the future, when (hopefully) more and more Lua bytecode data structures will be available directly from Flash (without having to copy them RAM as it happens now). It might also decrease the romfs memory footprint, but then again, it might not; it pretty much depends on the Lua programs included in the romfs.
2009-12-02 19:33:03 +00:00
def mkfs( dirname, outname, flist, mode, compcmd ):
# Try to create the output files
2008-07-29 11:08:54 +00:00
outfname = outname + ".h"
try:
outfile = file( outfname, "wb" )
except:
print "Unable to create output file"
return False
global _crtline, _numdata, _bytecnt
_crtline = ' '
_numdata = 0
_bytecnt = 0
# Generate headers
outfile.write( "// Generated by mkfs.py\n// DO NOT MODIFY\n\n" )
outfile.write( "#ifndef __%s_H__\n#define __%s_H__\n\n" % ( outname.upper(), outname.upper() ) )
outfile.write( "const unsigned char %s_fs[] = \n{\n" % ( outname.lower() ) )
# Process all files
for fname in flist:
if len( fname ) > maxlen:
print "Skipping %s (name longer than %d chars)" % ( fname, maxlen )
continue
2008-07-29 11:08:54 +00:00
# Get actual file name
realname = os.path.join( dirname, fname )
# Ensure it actually is a file
if not os.path.isfile( realname ):
print "Skipping %s ... (not found or not a regular file)" % fname
2008-07-29 11:08:54 +00:00
continue
# Try to open and read the file
try:
crtfile = file( realname, "rb" )
except:
outfile.close()
os.remove( outfname )
print "Unable to read %s" % fname
return False
EXPERIMENTAL FEATURE: romfs compression/precompilation. Added another "romfs" parameter to SConstruct. It can take one of 3 values: - "verbatim" (default): copy all the files from the eLua romfs/ directory to the binary image (exactly what happened until now) - "compress": use LuaSrcDiet (http://luaforge.net/projects/luasrcdiet, now included in the eLua distribution) to "compress" the source code by using different tricks (shorter identifiers, removing comments and EOLS, and so on). The output is still a compilable Lua file (although in most case it looks completely different from the original) which is copied in the eLua binary image instead of the original Lua file. The compression obtained by using this method seems to be very good in practice, thus it will save flash when needed. - "compile": use the eLua cross compiler (that must be built first by running 'scons -f cross-lua.py') to precompile the Lua files to bytecode, and write the bytecode to the image instead of the source file. This way, one can save both time (the scripts don't need to be compiled anymore) and RAM (for the same reason, especially since the Lua parser uses the stack a lot, which can lead to very nasty and hard to diagnose stack overflow bugs). It will be even more useful in the future, when (hopefully) more and more Lua bytecode data structures will be available directly from Flash (without having to copy them RAM as it happens now). It might also decrease the romfs memory footprint, but then again, it might not; it pretty much depends on the Lua programs included in the romfs.
2009-12-02 19:33:03 +00:00
# Do we need to process the file?
fextpart = ''
EXPERIMENTAL FEATURE: romfs compression/precompilation. Added another "romfs" parameter to SConstruct. It can take one of 3 values: - "verbatim" (default): copy all the files from the eLua romfs/ directory to the binary image (exactly what happened until now) - "compress": use LuaSrcDiet (http://luaforge.net/projects/luasrcdiet, now included in the eLua distribution) to "compress" the source code by using different tricks (shorter identifiers, removing comments and EOLS, and so on). The output is still a compilable Lua file (although in most case it looks completely different from the original) which is copied in the eLua binary image instead of the original Lua file. The compression obtained by using this method seems to be very good in practice, thus it will save flash when needed. - "compile": use the eLua cross compiler (that must be built first by running 'scons -f cross-lua.py') to precompile the Lua files to bytecode, and write the bytecode to the image instead of the source file. This way, one can save both time (the scripts don't need to be compiled anymore) and RAM (for the same reason, especially since the Lua parser uses the stack a lot, which can lead to very nasty and hard to diagnose stack overflow bugs). It will be even more useful in the future, when (hopefully) more and more Lua bytecode data structures will be available directly from Flash (without having to copy them RAM as it happens now). It might also decrease the romfs memory footprint, but then again, it might not; it pretty much depends on the Lua programs included in the romfs.
2009-12-02 19:33:03 +00:00
if mode == "compile" or mode == "compress":
fnamepart, fextpart = os.path.splitext( realname )
if mode == "compress":
newext = ".lua.tmp"
else:
newext = ".lc"
EXPERIMENTAL FEATURE: romfs compression/precompilation. Added another "romfs" parameter to SConstruct. It can take one of 3 values: - "verbatim" (default): copy all the files from the eLua romfs/ directory to the binary image (exactly what happened until now) - "compress": use LuaSrcDiet (http://luaforge.net/projects/luasrcdiet, now included in the eLua distribution) to "compress" the source code by using different tricks (shorter identifiers, removing comments and EOLS, and so on). The output is still a compilable Lua file (although in most case it looks completely different from the original) which is copied in the eLua binary image instead of the original Lua file. The compression obtained by using this method seems to be very good in practice, thus it will save flash when needed. - "compile": use the eLua cross compiler (that must be built first by running 'scons -f cross-lua.py') to precompile the Lua files to bytecode, and write the bytecode to the image instead of the source file. This way, one can save both time (the scripts don't need to be compiled anymore) and RAM (for the same reason, especially since the Lua parser uses the stack a lot, which can lead to very nasty and hard to diagnose stack overflow bugs). It will be even more useful in the future, when (hopefully) more and more Lua bytecode data structures will be available directly from Flash (without having to copy them RAM as it happens now). It might also decrease the romfs memory footprint, but then again, it might not; it pretty much depends on the Lua programs included in the romfs.
2009-12-02 19:33:03 +00:00
if fextpart == ".lua":
newname = fnamepart + newext
if mode == "compress":
print "Compressing %s to %s ..." % ( realname, newname )
else:
print "Cross compiling %s to %s ..." % ( realname, newname )
os.system( compcmd % ( newname, realname ) )
# TODO: this assumes that the cross compiler ended OK
crtfile.close()
try:
crtfile = file( newname, "rb" )
except:
outfile.close()
os.remove( outfname )
print "Unable to read %s" % newname
return False
if mode == "compile":
fnamepart, fextpart = os.path.splitext( fname )
fname = fnamepart + ".lc"
filedata = crtfile.read()
crtfile.close()
if fextpart == ".lua" and mode != "verbatim":
os.remove( newname )
EXPERIMENTAL FEATURE: romfs compression/precompilation. Added another "romfs" parameter to SConstruct. It can take one of 3 values: - "verbatim" (default): copy all the files from the eLua romfs/ directory to the binary image (exactly what happened until now) - "compress": use LuaSrcDiet (http://luaforge.net/projects/luasrcdiet, now included in the eLua distribution) to "compress" the source code by using different tricks (shorter identifiers, removing comments and EOLS, and so on). The output is still a compilable Lua file (although in most case it looks completely different from the original) which is copied in the eLua binary image instead of the original Lua file. The compression obtained by using this method seems to be very good in practice, thus it will save flash when needed. - "compile": use the eLua cross compiler (that must be built first by running 'scons -f cross-lua.py') to precompile the Lua files to bytecode, and write the bytecode to the image instead of the source file. This way, one can save both time (the scripts don't need to be compiled anymore) and RAM (for the same reason, especially since the Lua parser uses the stack a lot, which can lead to very nasty and hard to diagnose stack overflow bugs). It will be even more useful in the future, when (hopefully) more and more Lua bytecode data structures will be available directly from Flash (without having to copy them RAM as it happens now). It might also decrease the romfs memory footprint, but then again, it might not; it pretty much depends on the Lua programs included in the romfs.
2009-12-02 19:33:03 +00:00
2008-07-29 11:08:54 +00:00
# Write name, size, id, numpars
for c in fname:
_add_data( ord( c ), outfile )
_add_data( 0, outfile ) # ASCIIZ
size_l = len( filedata ) & 0xFF
size_h = ( len( filedata ) >> 8 ) & 0xFF
_add_data( size_l, outfile )
_add_data( size_h, outfile )
# Then write the rest of the file
for c in filedata:
_add_data( ord( c ), outfile )
# Report
print "Encoded file %s (%d bytes)" % ( fname, len( filedata ) )
# All done, write the final "0" (terminator)
_add_data( 0, outfile, False )
outfile.write( "};\n\n#endif\n" );
outfile.close()
print "Done, total size is %d bytes" % _bytecnt
2008-07-29 11:08:54 +00:00
return True
EXPERIMENTAL FEATURE: romfs compression/precompilation. Added another "romfs" parameter to SConstruct. It can take one of 3 values: - "verbatim" (default): copy all the files from the eLua romfs/ directory to the binary image (exactly what happened until now) - "compress": use LuaSrcDiet (http://luaforge.net/projects/luasrcdiet, now included in the eLua distribution) to "compress" the source code by using different tricks (shorter identifiers, removing comments and EOLS, and so on). The output is still a compilable Lua file (although in most case it looks completely different from the original) which is copied in the eLua binary image instead of the original Lua file. The compression obtained by using this method seems to be very good in practice, thus it will save flash when needed. - "compile": use the eLua cross compiler (that must be built first by running 'scons -f cross-lua.py') to precompile the Lua files to bytecode, and write the bytecode to the image instead of the source file. This way, one can save both time (the scripts don't need to be compiled anymore) and RAM (for the same reason, especially since the Lua parser uses the stack a lot, which can lead to very nasty and hard to diagnose stack overflow bugs). It will be even more useful in the future, when (hopefully) more and more Lua bytecode data structures will be available directly from Flash (without having to copy them RAM as it happens now). It might also decrease the romfs memory footprint, but then again, it might not; it pretty much depends on the Lua programs included in the romfs.
2009-12-02 19:33:03 +00:00