multiple crashes in lame

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

multiple crashes in lame

Agostino Sarubbo
Reply | Threaded
Open this post in threaded view
|

Re: multiple crashes in lame

Thomas Orgis
Am Wed, 28 Jun 2017 16:10:48 +0200
schrieb Agostino Sarubbo <[hidden email]>:

> https://blogs.gentoo.org/ago/2017/06/17/lame-global-buffer-overflow-in-ii_step_one-layer2-c
>
> https://blogs.gentoo.org/ago/2017/06/17/lame-global-buffer-overflow-in-iii_i_stereo-layer3-c
>
> https://blogs.gentoo.org/ago/2017/06/17/lame-stack-based-buffer-overflow-in-iii_i_stereo-layer3-c
>
> https://blogs.gentoo.org/ago/2017/06/17/lame-stack-based-buffer-overflow-in-iii_dequantize_sample-layer3-c

This is a good opportunity to finally get the mpglib fork in lame
replaced with libmpg123. I know we're short on developer time here …
but could someone at least point out what additions to the libmpg123
API (http://mpg123.org/api/) are needed to replace lame's fork? There
were needs for some frame analyser tool … worst case would be mapping
this part to libmpg123 API that simply gives you access the raw frame
data. The decoding aspect of mpglib, which contains the above-mentioned
bugs, can be replaced with less effort, I presume.


Alrighty then,

Thomas
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Lame-dev mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/lame-dev
Reply | Threaded
Open this post in threaded view
|

Re: multiple crashes in lame

Mp3 - Lame mailing list
In reply to this post by Agostino Sarubbo
Gio 29/6/17, Thomas Orgis <[hidden email]> ha scritto:

 
 This is a good opportunity to
 finally get the mpglib fork in lame
 replaced
 with libmpg123. I know we're short on developer time
 here …



Different point of view here.
I think this is the right time for lame to get rid of mpglib, following the old rule of "doing one thing, and doing it well".
Lame encodes audio into layer3 bitstreams, period.

On the other hand, the same level of features may be achieved using libsndfile, now that layer3 encoding is not encumbered by patents anymore and that library is going to support mpeg decoding.
See https://github.com/erikd/libsndfile/issues/258
so, when libsndfile will offer audio mpeg decoding (hopefully very soon), then lame will just link to libsndfile and we'll get the same ability to encode already encoded audio content.

Finally, IMHO lame should not concern about mpeg decoding, both mpg123 and madplay are already there, doing the same job.

Elio




------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Lame-dev mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/lame-dev
Reply | Threaded
Open this post in threaded view
|

Re: multiple crashes in lame

Thomas Orgis
Am Thu, 29 Jun 2017 13:38:32 +0000 (UTC)
schrieb eblanca76 via Lame-dev <[hidden email]>:

> Different point of view here.
> I think this is the right time for lame to get rid of mpglib,
> following the old rule of "doing one thing, and doing it well".

I do not oppose that. It's only that LAME folks adapted the mpglib to
do more things than just decoding (checking what the encoder produced
with some more insight). If we consider LAME rather finished now, the
need for these changes in mpglib might not be present anymore, or
interested people could port them into a stand-alone tool for
developers only.

> Lame encodes audio into layer3 bitstreams, period.

… and `mpg123 -s input.mpg | lame` re-encodes MPEG audio layer 1/2/3
into layer 3 bitstreams. I'd be fine with that (or `mpg123 -w tmp.wav
input.mpg && lame tmp.wav output.mp3`, of course).

> On the other hand, the same level of features may be achieved using
> libsndfile, now that layer3 encoding is not encumbered by patents
> anymore and that library is going to support mpeg decoding. See
> https://github.com/erikd/libsndfile/issues/258

Ah, finally! I faintly remember prodding Eric in the past (twice), but
he's too busy for replies or I just hopped into the spam folder. As
libsndfile is a lib for reading and writing and likely to end up
linking against lame itself for encoding, I would like to avoid lame
depending on libsndfile in return, though. As a distro maintainer, I
hate circular dependencies.

I didn't bother with a private github account yet (as I don't use it),
so I cannot comment to the libsndfile issue, I presume. I just feel a
little urge to remind people that libmpg123 also does fixed-point math
if you choose that kind of build and does it considerably faster than
libmad … you can choose between plain ARM and also NEON assembly with
proper floating point output, too (reminds me: would lame accept 32 bit
float input at some point?).

> Finally, IMHO lame should not concern about mpeg decoding, both
> mpg123 and madplay are already there, doing the same job.

+1 May need a major version bump to make it clear that feature set
changed.


Alrighty then,

Thomas
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Lame-dev mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/lame-dev
Reply | Threaded
Open this post in threaded view
|

Re: multiple crashes in lame

Tobias Damisch
>> Lame encodes audio into layer3 bitstreams, period.

I was once considering an attempt to a multithreaded lame fork, got
discouraged in the research process and didn't do it but remember
reading somewhere that lame, when re-encoding from mp3, does not
simply decode into raw pcm data and feed that to the encoder but uses
the already encoded data as a basis for calculations, possibly
resulting in better output quality.

I can't find a reference to this information now, but if it is true
this IMHO seems worth some consideration.

Regards,

Tobias


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Lame-dev mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/lame-dev
Reply | Threaded
Open this post in threaded view
|

Re: multiple crashes in lame

Thomas Orgis
Am Thu, 29 Jun 2017 23:28:56 +0200
schrieb Tobias Damisch <[hidden email]>:

> reading somewhere that lame, when re-encoding from mp3, does not
> simply decode into raw pcm data and feed that to the encoder but uses
> the already encoded data as a basis for calculations, possibly
> resulting in better output quality.

Hm … do we have somebody still alive being able to give us an overview
over that or do we need to dig into the (doubtlessly abundantly clear
and well-documented;-) code ourselves to find out? Such usage of a
decoder would of course justify containing perhaps some special frame
parser code in addition to the use of libmpg123 for decoding and ID3
parsing (for keeping the metadata).

I'd still like to avoid linking in libsndfile which in turn uses
libmp3lame in future, because of unclear build order of the two.


Alrighty then,

Thomas
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Lame-dev mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/lame-dev
Reply | Threaded
Open this post in threaded view
|

Re: multiple crashes in lame

Mp3 - Lame mailing list


Il 29/06/2017 23:42, Thomas Orgis ha scritto:
[...]
> I'd still like to avoid linking in libsndfile which in turn uses
> libmp3lame in future, because of unclear build order of the two.

It probably would be a mess (I have no experience in package maintenance), but put it simple:
libsndfile has a single part to build (the library itself), lame has two (the library and the
frontend). So, the order would be the lame library (no depencencies), then the libsndfile library
(which needs libmp3lame) and then the lame frontend (which needs the full libsndfile).

Now, adding more pieces to this scenario may make it harder (it will) but perhaps (at this moment)
we are looking a too far future (well, far from Agostino's input).

Elio


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Lame-dev mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/lame-dev
Reply | Threaded
Open this post in threaded view
|

Re: multiple crashes in lame

Thomas Orgis
Am Fri, 30 Jun 2017 00:52:22 +0200
schrieb Elio Blanca via Lame-dev <[hidden email]>:

> It probably would be a mess (I have no experience in package
> maintenance), but put it simple: libsndfile has a single part to
> build (the library itself), lame has two (the library and the
> frontend). So, the order would be the lame library (no depencencies),
> then the libsndfile library (which needs libmp3lame) and then the
> lame frontend (which needs the full libsndfile).

Yes, there are ways to achieve this, but it is a pain especially if you
want to support on-the-fly builds of the things (source-based distros
like Gentoo or Source Mage, what I am dealing with, or also BSD ports).
You need to artificially split up upstream packages into these pieces.
This happens anyway for binary distros like Debian, because you gain
some flexibility/optimisation of disk usage in the end. Source-based
usually coincides with small user base and rather small developer count
… getting a wild mix of compilers and libs working with rolling release
from sources is tricky enough.

Anyone trying to just even build the software by hand to get some
proper versions on some special system that does not package them will
curse you for inflicting circular dependencies. While consumption of
binary packages from a distro is the norm nowadays, Open Source
software should always offer the option to just grab the source and
play with it, with a clear order of what depends on what.

That being said … one could separate lame frontend from lame library
and serve them as two independent upstream packages. But it's still
more work taking care of two lame packages (with diverging versions)
than one.

</packager_maintainer_rant>

And, to get back on topic: Does libmp3lame feature the option of
transcoding from MP3 to MP3 or is that only in the frontend? I guess at
least the transcoding of arbitrary libsndfile formats to MP3 would be
left to the frontend only. But then … a generic libsndfile frontend
might be a more proper solution. People who actually use lame on the
command line might just be crafty enough to decode other formats to WAV
first with other tools … the Unix way and all that …


Alrighty then,

Thomas
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Lame-dev mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/lame-dev
Reply | Threaded
Open this post in threaded view
|

Re: multiple crashes in lame

Mp3 - Lame mailing list
In reply to this post by Thomas Orgis

De: Thomas Orgis <[hidden email]>
Para: [hidden email] 
Enviadas: Quinta-feira, 29 de Junho de 2017 13:00
Assunto: Re: [Lame-dev] multiple crashes in lame

> I didn't bother with a private github account yet (as I don't use it),
> so I cannot comment to the libsndfile issue, I presume. I just feel a
> little urge to remind people that libmpg123 also does fixed-point math
> if you choose that kind of build and does it considerably faster than
> libmad … you can choose between plain ARM and also NEON assembly 
Done!
Regards; -- 
Roberto José de AmorimMSc in Computer Science
Columbia University - New York


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Lame-dev mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/lame-dev
Reply | Threaded
Open this post in threaded view
|

Re: multiple crashes in lame

bouvigne
In reply to this post by Thomas Orgis
On 2017-06-29 23:42, Thomas Orgis wrote:

>> reading somewhere that lame, when re-encoding from mp3, does not
>> simply decode into raw pcm data and feed that to the encoder but uses
>> the already encoded data as a basis for calculations, possibly
>> resulting in better output quality.
>
> Hm … do we have somebody still alive being able to give us an overview
> over that or do we need to dig into the (doubtlessly abundantly clear
> and well-documented;-) code ourselves to find out? Such usage of a
> decoder would of course justify containing perhaps some special frame
> parser code in addition to the use of libmpg123 for decoding and ID3
> parsing (for keeping the metadata).

Gentlemen,

When re-encoding from MP3, LAME does not make any use of algorithmical
data from the previously encoded MP3 (so you can go
MP3->decodetofloatingpoint->encodetoMP3).

The specific points that Lame is using in the decoding library are:
*gapless decoding (based on LAME/INFO tag)
*gapless decoding based on stitching several encoded MP3 (encoded with
the --nogap option). That could be considered as deprecated (because in
reality only the LAME/INFO method is used by other decoders)
*carrying ID3/ID3v2 tags
*hooks for the MP3x frame analyser. Even if MP3x is a bit tricky to
build, it is very valuable for people working on the encoder.


--
Gabriel Bouvigne

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Lame-dev mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/lame-dev
Reply | Threaded
Open this post in threaded view
|

Re: multiple crashes in lame

Thomas Orgis
Am Fri, 30 Jun 2017 13:56:11 +0200
schrieb [hidden email]:

> When re-encoding from MP3, LAME does not make any use of algorithmical
> data from the previously encoded MP3 (so you can go
> MP3->decodetofloatingpoint->encodetoMP3).

Great. Also I just verified that current lame is happy with floating
point WAV input (remembered differently).

> The specific points that Lame is using in the decoding library are:
> *gapless decoding (based on LAME/INFO tag)

Done by libmpg123 (actually one of the main points that got me hacking
on mpg123 in the first place). WAVs created by `mpg123 -w output.wav
input.mp3` are gapless if info is present.

> *gapless decoding based on stitching several encoded MP3 (encoded with
> the --nogap option). That could be considered as deprecated (because in
> reality only the LAME/INFO method is used by other decoders)

I only very faintly remember --nogap, but not actually using it.

> *carrying ID3/ID3v2 tags

Libmpg123 gives you those, to some extent (parsed to UTF-8 strings).
Maybe we need some added hooks to extract the raw ID3v2 tag(s) for lame
… as libmpg123 does not deliver unsupported parts of ID3v2. A separate
tool that copies metadata from one MP3 to another one might be a more
proper way. There are better metadata parsers and writers out there
(also supporting a variety of tag formats).

> *hooks for the MP3x frame analyser. Even if MP3x is a bit tricky to
> build, it is very valuable for people working on the encoder.

I'm very much interested in what those hooks are exactly to add them to
the libmpg123 API.


Alrighty then,

Thomas
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Lame-dev mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/lame-dev
Reply | Threaded
Open this post in threaded view
|

Re: multiple crashes in lame

Robert Hegemann
Hi,

here is what I remember, LAME uses its own mpeg decoder fork for:

+ lame: for transcoding, could use any other library for this task
+ MP3x: hooks to allow inspection of MP3 data, input and output
+ libmp3lame: replaygain accurate, values calculated on decoded mp3 data  
while encoding
+ dshow/activeX filter: *not sure anymore* for the decoding part?

Ciao Robert

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Lame-dev mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/lame-dev