@CoderSupreme@programming.dev to Technology@lemmy.worldEnglish • 2 years agoAI language models can exceed PNG and FLAC in lossless compression, says studyarstechnica.comexternal-linkmessage-square12fedilinkarrow-up191arrow-down17cross-posted to: technology@beehaw.org
arrow-up184arrow-down1external-linkAI language models can exceed PNG and FLAC in lossless compression, says studyarstechnica.com@CoderSupreme@programming.dev to Technology@lemmy.worldEnglish • 2 years agomessage-square12fedilinkcross-posted to: technology@beehaw.org
minus-squareferretlinkfedilinkEnglish11•2 years agoHow can they be lossless? Isn’t a neural network inherently lossy?
minus-square@Bogasse@lemmy.mllinkfedilinkEnglish6•edit-22 years agoI suppose the compression process looks like this : call the model to predict the most probable next tokens (this is deterministic) encode next tokens by with its ranking in model prediction If the model is good at predicting what the next token is, I suppose you need only 2bits to encode each token (for any of the top 4 predictions).
minus-square@9point6@lemmy.worldlinkfedilinkEnglish6•2 years agoLossless in terms of compression is being able to reconstruct the original bits of a piece of media exactly from its compressed bits. The thing that I’m wondering is how reliable this is
How can they be lossless? Isn’t a neural network inherently lossy?
I suppose the compression process looks like this :
If the model is good at predicting what the next token is, I suppose you need only 2bits to encode each token (for any of the top 4 predictions).
Lossless in terms of compression is being able to reconstruct the original bits of a piece of media exactly from its compressed bits.
The thing that I’m wondering is how reliable this is