Discussion:
Read an embedded resource from a .NET assembly without reflexion
(too old to reply)
Anton Shepelev
2017-09-26 10:24:03 UTC
Permalink
Hello, all

For the sake of speed and simplicity, I want to read
an embedded resource from a .NET assembly file with-
out loading this assembly via reflexion. Can I, for
example, read the assembly via File.ReadAllBytes()
and locate an embedded text reasource in the byte
array or stream?
--
() ascii ribbon campaign - against html e-mail
/\ http://preview.tinyurl.com/qcy6mjc [archived]
Anton Shepelev
2017-09-26 12:06:54 UTC
Permalink
For the sake of speed and simplicity, I want to
read an embedded resource from a .NET assembly file
without loading this assembly via reflexion. Can
I, for example, read the assembly via File.
ReadAllBytes() and locate an embedded text rea-
source in the byte array or stream?
I have found text resources in the .dll with a text
editor, but I don't know the layout conventions,
such as the mapping of identifiers to resources, the
storage of their sizes, separators, &c. Where can
those be documeted?
--
() ascii ribbon campaign - against html e-mail
/\ http://preview.tinyurl.com/qcy6mjc [archived]
Marcel Mueller
2017-09-26 19:46:38 UTC
Permalink
Post by Anton Shepelev
Hello, all
For the sake of speed and simplicity, I want to read
an embedded resource from a .NET assembly file with-
out loading this assembly via reflexion. Can I, for
example, read the assembly via File.ReadAllBytes()
and locate an embedded text reasource in the byte
array or stream?
Of course, you can reinvent the wheel and implement the .NET assembly
loader by yourself. Whether you end up with a faster or slower solution
is another question. Including development time you will probably be
significantly slower.

Concerning speed you could try ReflectionOnlyLoadFrom.


Marcel
Anton Shepelev
2017-09-27 09:06:00 UTC
Permalink
Post by Marcel Mueller
For the sake of speed and simplicity, I want to
read an embedded resource from a .NET assembly
file without loading this assembly via reflexion.
Can I, for example, read the assembly via File.
ReadAllBytes() and locate an embedded text rea-
source in the byte array or stream?
Of course, you can reinvent the wheel and implement
the .NET assembly loader by yourself.
Inveting a thing is a great way to understand how it
works and to avoid many ineffective dependenices,
when one is re-using 1% of some huge and bloated
framework to perform a task that takes 100 lines of
code.

I do not need an assembly loader. If fact, I want
to avoid it together with the overhead of the cre-
ation of a separate application domain.
Post by Marcel Mueller
Whether you end up with a faster or slower solution
is another question. Including development time
you will probably be significantly slower.
I have not finished my manual binary parser, but
from what I have understood it should be much faster
than ReflectionOnlyLoad into a separate application
domain.
--
() ascii ribbon campaign - against html e-mail
/\ http://preview.tinyurl.com/qcy6mjc [archived]
Arne Vajhøj
2017-09-28 00:39:41 UTC
Permalink
Post by Anton Shepelev
Post by Marcel Mueller
For the sake of speed and simplicity, I want to
read an embedded resource from a .NET assembly
file without loading this assembly via reflexion.
Can I, for example, read the assembly via File.
ReadAllBytes() and locate an embedded text rea-
source in the byte array or stream?
Of course, you can reinvent the wheel and implement
the .NET assembly loader by yourself.
Inveting a thing is a great way to understand how it
works and to avoid many ineffective dependenices,
when one is re-using 1% of some huge and bloated
framework to perform a task that takes 100 lines of
code.
You seem to be already on 200 lines of code.

And what does it matter that .NET is big? All your
users must have it installed already.
Post by Anton Shepelev
I do not need an assembly loader. If fact, I want
to avoid it together with the overhead of the cre-
ation of a separate application domain.
Yo do not need to create a new application domain.
Post by Anton Shepelev
Post by Marcel Mueller
Whether you end up with a faster or slower solution
is another question. Including development time
you will probably be significantly slower.
I have not finished my manual binary parser, but
from what I have understood it should be much faster
than ReflectionOnlyLoad into a separate application
domain.
How much will the execution time of the real
application be reduced?

I don't expect any real application to spend
much time doing stuff like this. Meaning that
even large relative improvements if the specific
sub-functionality has little impact on the
overall performance.

The cost of KLOC's over the application life cycle
on the other hand is very real.

Arne
Marcel Mueller
2017-09-28 21:02:18 UTC
Permalink
Post by Arne Vajhøj
Post by Anton Shepelev
I do not need an assembly loader. If fact, I want
to avoid it together with the overhead of the cre-
ation of a separate application domain.
Yo do not need to create a new application domain.
If you want a scalable solution you won't come around that. .NET has no
option to /unload/ a previously loaded assembly without discarding the
entire application domain.


Marcel
Arne Vajhøj
2017-09-29 01:10:41 UTC
Permalink
Post by Marcel Mueller
Post by Arne Vajhøj
I  do  not need an assembly loader.  If fact, I want
to avoid it together with the overhead of  the  cre-
ation of a separate application domain.
Yo do not need to create a new application domain.
If you want a scalable solution you won't come around that. .NET has no
option to /unload/ a previously loaded assembly without discarding the
entire application domain.
That is true.

Arne
Anton Shepelev
2017-09-29 10:31:56 UTC
Permalink
Post by Arne Vajhøj
Inveting a thing is a great way to understand how
it works and to avoid many ineffective depen-
denices, when one is re-using 1% of some huge and
bloated framework to perform a task that takes 100
lines of code.
You seem to be already on 200 lines of code.
Indeed. The inheritance mechanim of C# has proven
clumsy and inadequate for so trivial a task as im-
plementing a Stream that works with a subsection of
an original stream. A slightly fixed version is
here:

https://stackoverflow.com/a/46443269/2862241

But my point is that using the full power of your
tiny custom module is much better design than using
1% of the functionality of some huge framework, such
as Cecil, which is simply too great for so small a
task as the reading of an embedded resource.

Perfection is the supremacy of the whole over the
sum of its parts. With Cecil the sum of the parts
will be larger than the whole -- a negative synergy.
Post by Arne Vajhøj
And what does it matter that .NET is big? All your
users must have it installed already.
Cecil is not part of standard .NET, is it? Reliance
on a third-party library is more costly than re-
liance on stardard functionality, whether big or
small.

But I dislike bloat in itself. I will not, for ex-
ample, use a Dictionary<> for the simple mapping the
values of an enum to strings. A always try to use
the simplest possible mechanism that is elegant and
sufficiently flexible.
Post by Arne Vajhøj
I do not need an assembly loader. If fact, I want
to avoid it together with the overhead of the cre-
ation of a separate application domain.
Yo do not need to create a new application domain.
I think I do. I am writing a mechanism to check
compatibility between .NET assemblies based on ver-
sion numbers and dependencies. We have an internal
convention that the minor version number denotes
chages that keep backwards compatibility and the ma-
jor one chages that violate it.

If some assembly fail the test the mechanism shall
unload it, to let the user try another version of
the same assembly without restarting the program.
Whether such mechanism is justified is another very
good question, and if you can suggest better or sim-
pler alternatives, please do so in a separate
thread. I might start this thread myself when time
permits.
Post by Arne Vajhøj
I have not finished my manual binary parser, but
from what I have understood it should be much
faster than ReflectionOnlyLoad into a separate ap-
plication domain.
How much will the execution time of the real appli-
cation be reduced?
If we decide to run the test at application start,
the impact will be significant: my custom parser ex-
ctracts a single text resource four times faster
than Cesil, for example. On the other hand, speed
will be unimportant if the work is delegated to a
package manager.
Post by Arne Vajhøj
The cost of KLOC's over the application life cycle
on the other hand is very real.
I agree. But so is the cost of dependencies on
third-party libraries (Cecil). As to the loading of
assemblies, I find it overkilll for so simple a task
as the reading of a text resource.
--
() ascii ribbon campaign - against html e-mail
/\ http://preview.tinyurl.com/qcy6mjc [archived]
Arne Vajhøj
2017-09-30 18:41:53 UTC
Permalink
Post by Anton Shepelev
Post by Arne Vajhøj
Inveting a thing is a great way to understand how
it works and to avoid many ineffective depen-
denices, when one is re-using 1% of some huge and
bloated framework to perform a task that takes 100
lines of code.
You seem to be already on 200 lines of code.
Indeed. The inheritance mechanim of C# has proven
clumsy and inadequate for so trivial a task as im-
plementing a Stream that works with a subsection of
an original stream. A slightly fixed version is
https://stackoverflow.com/a/46443269/2862241
Extending Stream is a bit cumbersome.

We need System.IO.FilterStream class similar to
Java's java.io.FilterInputStream.

:-)
Post by Anton Shepelev
But my point is that using the full power of your
tiny custom module is much better design than using
1% of the functionality of some huge framework, such
as Cecil, which is simply too great for so small a
task as the reading of an embedded resource.
Perfection is the supremacy of the whole over the
sum of its parts. With Cecil the sum of the parts
will be larger than the whole -- a negative synergy.
I tend to distinguish between hobby programming and
work programming here.

For hobby programming it is a valid goal to try and
make the the code as efficient/elegant/beautiful
as possible.

For work programming it is just business nothing
personal. Whatever works and minimizes the TCO.
Maintaining code cost money. If an external library
can provide the functionality (and that library
seems to have a future!) then it saves money. If
that library also provides a lot of not needed
functionality does not matter.
Post by Anton Shepelev
Post by Arne Vajhøj
And what does it matter that .NET is big? All your
users must have it installed already.
Cecil is not part of standard .NET, is it?
No. That remark was for the Reflection solution.
Post by Anton Shepelev
Reliance
on a third-party library is more costly than re-
liance on stardard functionality, whether big or
small.
Minimizing third party libraries are good.

But it gets bad when one start to write code
to do the same as the library.
Post by Anton Shepelev
But I dislike bloat in itself. I will not, for ex-
ample, use a Dictionary<> for the simple mapping the
values of an enum to strings. A always try to use
the simplest possible mechanism that is elegant and
sufficiently flexible.
As long as it reduces the KLOC's to maintain then I am
all for it.

I become very skeptical when it increases them.
Post by Anton Shepelev
Post by Arne Vajhøj
I have not finished my manual binary parser, but
from what I have understood it should be much
faster than ReflectionOnlyLoad into a separate ap-
plication domain.
How much will the execution time of the real appli-
cation be reduced?
If we decide to run the test at application start,
the impact will be significant: my custom parser ex-
ctracts a single text resource four times faster
than Cesil, for example. On the other hand, speed
will be unimportant if the work is delegated to a
package manager.
If that does not have any bad side effects then that may
be the best solution. Or cache test results in some
way.
Post by Anton Shepelev
Post by Arne Vajhøj
The cost of KLOC's over the application life cycle
on the other hand is very real.
I agree. But so is the cost of dependencies on
third-party libraries (Cecil).
That also has some cost.

But the biggest problem is whether the libraries will
stay around.

I would not worry about Cecil for that. There are a lot
of stuff depending on it.
Post by Anton Shepelev
As to the loading of
assemblies, I find it overkilll for so simple a task
as the reading of a text resource.
But does it really matter - that is the question.

Arne
Arne Vajhøj
2017-09-30 19:13:00 UTC
Permalink
Post by Anton Shepelev
Post by Arne Vajhøj
I do not need an assembly loader. If fact, I want
to avoid it together with the overhead of the cre-
ation of a separate application domain.
Yo do not need to create a new application domain.
I think I do. I am writing a mechanism to check
compatibility between .NET assemblies based on ver-
sion numbers and dependencies. We have an internal
convention that the minor version number denotes
chages that keep backwards compatibility and the ma-
jor one chages that violate it.
If some assembly fail the test the mechanism shall
unload it, to let the user try another version of
the same assembly without restarting the program.
Unloading requires a separate application domain.

But you may be able to not create and remove one
for each assembly.
Post by Anton Shepelev
Whether such mechanism is justified is another very
good question, and if you can suggest better or sim-
pler alternatives, please do so in a separate
thread. I might start this thread myself when time
permits.
It seems like you are trying to implement something
similar to OSGi bundles.

If you are not familiar with it then OSGi is a java
module thingy.

Basically bundles export specific versions of things
like:

Export-Package: org.foo; version=1.2.1.1

and import specific versions of things like:

Import-Package: org.bar; version=1.3.2.1

for requiring version >= 1.3.2.1 and:

Import-Package: org.bar; version=[1.3.2.1,1.99.99.99]

for requiring 1.3.2.1 <= version >= 1.99.99.99.

There exist an OSGi port to .NET:

https://osgi.codeplex.com/
https://sourceforge.net/projects/osginet/

But I have never heard about anyone using them, so I
will *not* recommend them.

Microsofts approach is a bit different.

I think their solution for the problem is MAF
(Managed Addin Framework).

But it does not work like what you describe. And
I have a strong feeling that you will not like MAF
(it is a bit heavy).

So no revolutionary idea.

I would probably have gone for the reflection
solution and worked around any encountered
performance issues.

Arne

Arne Vajhøj
2017-09-26 19:49:18 UTC
Permalink
Post by Anton Shepelev
For the sake of speed and simplicity, I want to read
an embedded resource from a .NET assembly file with-
out loading this assembly via reflexion. Can I, for
example, read the assembly via File.ReadAllBytes()
and locate an embedded text reasource in the byte
array or stream?
I would think that Mono.Cecil could do it.

Arne
Anton Shepelev
2017-09-27 08:54:53 UTC
Permalink
For the sake of speed and simplicity, I want to
read an embedded resource from a .NET assembly file
without loading this assembly via reflexion. Can
I, for example, read the assembly via File.
ReadAllBytes() and locate an embedded text rea-
source in the byte array or stream?
Here is the simplest thing that works:

using System;
using System.IO;
using System.Collections.Generic;
using System.Text;
using System.Resources;

namespace ResUtil
{
class ShiftedStream: Stream
{ Stream Base;
long Orig;

public override bool CanSeek
{ get
{ return Base.CanSeek; }
}

public override bool CanRead
{ get
{ return Base.CanRead; }
}

public override bool CanTimeout
{
get { return Base.CanTimeout; }
}

public override Boolean CanWrite {
get {
return Base.CanWrite;
}
}

public override long Position
{ get
{ return Base.Position - Orig; }
set
{ Base.Position = value + Orig; }
}

public override Int64 Length {
get { return Base.Length - Orig; }
}

public override void SetLength(Int64 value)
{ Base.SetLength( value + Orig ); }

public override Int32 Read(Byte[] buffer,Int32 offset,Int32 count)
{ return Base.Read( buffer, offset, count ); }

public override void Write(Byte[] buffer,Int32 offset,Int32 count)
{ Base.Write( buffer, offset, count ); }

public ShiftedStream( Stream s )
{ Base = s;
Orig = s.Position;
}

public override long Seek( long ofs, SeekOrigin orig )
{ if( orig == SeekOrigin.Begin )
{ ofs = ofs + Orig; }
return Base.Seek( ofs, orig );
}

public override void Flush()
{ Base.Flush(); }
}

public static class BinResReader
{ // TODO: a more generic overload accepting a stream instead of the file path
public static void Get
( string file, string resName, out string type, out byte[] data )
{ FileStream s;
ResourceReader rr;
long offset;
bool found;

type = null;
data = null;
s = new FileStream( file, FileMode.Open );
while( true )
{ if( !FindResHeader( s ) )
{ break; }
offset = s.Position;
try
{ rr = new ResourceReader( new ShiftedStream( s ) ); }
catch( BadImageFormatException ) // no resource here
{ continue; }

found = true;
// WARN: for the last resource in the file <data> will be read
// through till the end of file.
try
{ rr.GetResourceData( resName, out type, out data ); }
catch // resource res may not be present
{ found = false; }
rr.Dispose();
if( found )
{ break; }
s.Position = offset;
s.ReadByte();
if( !s.CanRead )
{ break; }
}
s.Dispose();
}

public static string GetStr( string file, string resName )
{ return GetStr( file, resName, Encoding.ASCII ); }

public static string GetStr( string file, string resName, Encoding enc )
{ byte[] data;
string text;
string type;
int count;
int index;
int len;
text = null;
Get( file, resName, out type, out data );
if( data != null )
{ switch( type )
{ case "ResourceTypeCode.ByteArray":
count = GetIntBE( data );
index = 4;
break;
case "ResourceTypeCode.String":
len = FindLen( data );
index = UsedBytes( len );
count = len - index;
break;
default: throw new Exception( "Cannot convert resource type " +
type + " to string.");
}
text = enc.GetString( data, index, count );
}
return text;
}

// Since I don't know the length of the length prefix, I must perform
// idiotic C-like acrobatics:
static int FindLen( byte[] data )
{ for( int i = 0; i < data.Length; i++ )
{ if( data[i] == 0 )
{ return i; }
}
return data.Length;
}

static int UsedBytes( int len )
{ int maxLen = Byte.MaxValue;
int i = 1;
while( ( len - i ) > maxLen )
{ i++;
maxLen = maxLen * Byte.MaxValue;
}
return i;
}

static bool FindResHeader( Stream s )
{ int b;
bool found = false;
int looking;

looking = 0;

while( true )
{ b = s.ReadByte();
if( b == -1 )
{ break; }
Repeat: // How would you write it without the goto?
if( b == MagicBytes[looking] )
{ if( looking == 3 )
{ found = true;
s.Seek( -4, SeekOrigin.Current );
break;
}
else
{ looking++; }
}
else
{ if( looking > 0 ) { looking = 0; goto Repeat; } }
}
return found;
}

const int ByteMagn = Byte.MaxValue + 1; // Just in case...

static int GetIntBE( byte[] bytes )
{ int res;
res = 0;
for( int i = 3; i >= 0; i-- )
{ Gorner( ref res, ByteMagn, bytes[i] ); }
return res;
}

static void Gorner( ref int value, int @base, int coef )
{ value = value * @base + coef; }

static int[] MagicBytes;
static BinResReader()
{
uint magic;
MagicBytes = new int[4];
magic = (uint)ResourceManager.MagicNumber;
for( int i = 0; i < 4; i++ )
{ MagicBytes[i] = (int)( magic % ByteMagn );
magic = magic / ByteMagn;
}
}
}
}

It may be improved in several ways, including:

a. using the name of the embedded resource file
in search, in addition to the name of the re-
source,
b. reading several resources in one pass.
--
() ascii ribbon campaign - against html e-mail
/\ http://preview.tinyurl.com/qcy6mjc [archived]
Loading...