GIT 457d9a9e5850477d636d79bc6f79df7b80954e23 git+ssh://master.kernel.org/pub/scm/linux/kernel/git/herbert/cryptodev-2.6.git commit 457d9a9e5850477d636d79bc6f79df7b80954e23 Author: Michal Ludvig Date: Mon Jul 17 08:14:58 2006 +1000 [CRYPTO] padlock-sha: TFMs don't need to be static TFMs are local variables. No need to declare them static. After all one is enough. Signed-off-by: Michal Ludvig Signed-off-by: Herbert Xu commit 53996ef2ea4022e872c0e1b1ef68e189bb258084 Author: Michal Ludvig Date: Mon Jul 17 08:10:45 2006 +1000 [CRYPTO] padlock: Helper module padlock.ko Compile a helper module padlock.ko that will try to autoload all configured padlock algorithms. This also provides backward compatibility with the ancient times before padlock.ko was renamed to padlock-aes.ko Signed-off-by: Michal Ludvig Signed-off-by: Herbert Xu commit d40634ed28bcf209b79cf86d895176235299c948 Author: Adrian Bunk Date: Sat Jul 15 11:31:25 2006 +1000 [CRYPTO] padlock-sha: Make 2 functions static This patch makes two needlessly global functions static. Signed-off-by: Adrian Bunk Signed-off-by: Herbert Xu commit f09bb1382634fc2081353055587ba956d7a97f41 Author: Michal Ludvig Date: Wed Jul 12 12:29:38 2006 +1000 [CRYPTO] padlock: Driver for SHA1 / SHA256 algorithms Support for SHA1 / SHA256 algorithms in VIA C7 processors. Signed-off-by: Michal Ludvig Signed-off-by: Herbert Xu commit dc0cf80fc0eb25203949a3bf01ee4beeba8c2842 Author: Herbert Xu Date: Sat Jul 15 10:23:49 2006 +1000 [CRYPTO] padlock: Update private header file PADLOCK_CRA_PRIORITY is shared between padlock-aes and padlock-sha so it should be in the header. On the other hand "struct cword" is only used in padlock-aes.c so it's unnecessary to have it in padlock.h Signed-off-by: Michal Ludvig Signed-off-by: Herbert Xu commit 49c1e214623b59f16f0c5b1f559c01ef950524d4 Author: Herbert Xu Date: Wed Jul 12 10:35:33 2006 +1000 [CRYPTO] tcrypt: Use test_hash for crc32c Now that crc32c has been fixed to conform with standard digest semantics, we can use test_hash for it. I've turned the last test into a chunky test. Signed-off-by: Herbert Xu commit a69adadc450f228945b8a437afe9451d13c885fc Author: Herbert Xu Date: Wed Jul 12 10:34:02 2006 +1000 [CRYPTO] crc32c: Fixed reversed endian annotation The input rather than the internal state is supposed to be little-endian. Signed-off-by: Herbert Xu commit a5c0ecd24a862d864a87abb77ded77cc18f53c5e Author: Herbert Xu Date: Tue Jul 11 14:48:51 2006 +1000 [CRYPTO] api: Fix potential cra_users leak in crypto_remove_spawns When we walk the spawns list we keep adding the cra_users list of each entry that we process to the end of the list. If the entry we process happens to be the last of the spawns list, the newly added entries will not be processed. Thie patch fixes it by changing n if the it is the last entry of spawns. Signed-off-by: Herbert Xu commit f0dc6bb308156944187fddb2d87ed1f94e72dbc7 Author: Herbert Xu Date: Tue Jul 11 12:00:32 2006 +1000 [CRYPTO] api: Remove spawns of eclipsed algorithms When registering a new algorithm it may eclipse existing ones in the sense that if anyone looks up the new algorithm's cra_name they will (or may) get it instead of the existing algorithm. Technically, that they have the same cra_name but the new algorithm has a higher or equal priority. In this case, we should remove all spawns of the eclipsed algorithms so that the new algorithm gets a chance to be spawned in place of the existing one. This may not always happen due to type/mask restrictions. However, in that case deleting the existing spawn will simply result in the existing algorithm being spawned again which is harmless. Signed-off-by: Herbert Xu commit 54b0d67b91b900e58226923a6eae7555dc7b231c Author: Herbert Xu Date: Mon Jul 10 16:21:06 2006 +1000 [CRYPTO] digest: Fix dit_final typo and remove dia_digest The choice of dia_final versus final for dit_final was reversed. This patch fixes that. It also removes dia_digest since it turns out that the only potential user for it, padlock-sha + IPsec, wouldn't work because it just so happens that for ESP the hash offset is off the encryption offset by 24 bytes for AES. Since padlock requires the input to be 16-byte aligned, at least one of them (or both) will have to cope with copying. If something else needs it in future we can always add it back. Signed-off-by: Herbert Xu commit dade052b008cbcf5390e2f539fde787ffa7cce8f Author: Herbert Xu Date: Sun Jul 9 14:49:42 2006 +1000 [CRYPTO] digest: Store temporary digest in tfm When the final result location is unaligned, we store the digest in a temporary buffer before copying it to the final location. Currently that buffer sits on the stack. This patch moves it to an area in the tfm, just like the CBC IV buffer. Signed-off-by: Herbert Xu commit 1e0531a99081c99191a795ca00d525563a27283f Author: Herbert Xu Date: Sun Jul 9 13:35:30 2006 +1000 [CRYPTO] api: Get rid of flags argument to setkey Now that the tfm is passed directly to setkey instead of the ctx, we no longer need to pass the &tfm->crt_flags pointer. This patch also gets rid of a few unnecessary checks on the key length for ciphers as the cipher layer guarantees that the key length is within the bounds specified by the algorithm. Rather than testing dia_setkey every time, this patch does it only once during crypto_alloc_tfm. The redundant check from crypto_digest_setkey is also removed. Signed-off-by: Herbert Xu commit d96818ae02143a8278727fcbeb25f56798f8faea Author: Herbert Xu Date: Sun Jul 9 12:01:58 2006 +1000 [CRYPTO] crc32c: Fix unconventional setkey usage The convention for setkey is that once it is set it should not change, in particular, init must not wipe out the key set by it. In fact, init should always be used after setkey before any digestion is performed. The only user of crc32c that sets the key is tcrypt. This patch adds the necessary init calls there. Signed-off-by: Herbert Xu commit b58009f35d88fa92ed40e3c0d7fcd3d5e8c3da94 Author: Herbert Xu Date: Sun Jul 9 10:35:49 2006 +1000 [CRYPTO] padlock: Add compatibility alias after rename Whenever we rename modules we should add an alias to ensure that existing users can still locate the new module. This patch also gets rid of the now unused module function prototypes from padlock.h. Signed-off-by: Herbert Xu commit c0b1d90220bfdff75e6a1069db18138fee6698cc Author: Michal Ludvig Date: Sun Jul 9 10:30:02 2006 +1000 [CRYPTO] padlock: Get rid of padlock-generic.c Merge padlock-generic.c into padlock-aes.c and compile AES as a standalone module. We won't make a monolithic padlock.ko with all supported algorithms, instead we'll compile each driver into its own module. Signed-off-by: Michal Ludvig Signed-off-by: Herbert Xu commit 398f9a31378652f4318feb845514b252ff2da99b Author: Michal Ludvig Date: Sun Jul 9 09:02:24 2006 +1000 [CRYPTO] api: Add missing accessors for new crypto_alg fields Add missing accessors for cra_driver_name and cra_priority. Signed-off-by: Michal Ludvig Signed-off-by: Herbert Xu commit c5c35b328c7ca5783b8abcbfaf3a80875ae9a0a8 Author: Michal Ludvig Date: Sun Jul 9 08:59:38 2006 +1000 [CRYPTO] sha: Add module aliases for sha1 / sha256 Crypto modules should be loadable by their .cra_driver_name, so we should make MODULE_ALIAS()es with these names. This patch adds aliases for SHA1 and SHA256 only as that's what we need for PadLock-SHA driver. Signed-off-by: Michal Ludvig Signed-off-by: Herbert Xu commit b4240dc93a6bb83a97a11428bb61520fd9805984 Author: Herbert Xu Date: Mon Jul 3 11:30:45 2006 +1000 [CRYPTO] api: Retry when modules are removed The module loading/unloading semantics here are such that as long as you load a replacement before unloading a module, e.g., modprobe md5-new rmmod -w md5 no operations on md5 should fail. It turns out that there are a few cases where this assumption fails for crypto instances. This patch addresses these cases by letting the API retry the operations if it fails because an underlying module has been removed. If no replacement module is loaded, then the retry will terminate. More specifically, each time a crypto_mod_get fails we mark the affected algorithm and the instances leading to this as dying. Dying algorithms are skipped by all further lookups. So eventually all algorithms related to an unloading module will be marked as dying which terminates the retry process. At this point a new larval would be created which will succeed if a replacement module is available. Signed-off-by: Herbert Xu commit 8b0600c41e5b9495675af9cdf42f90c7d436d903 Author: Herbert Xu Date: Mon Jul 3 11:08:13 2006 +1000 [CRYPTO] api: Avoid refilling larval->adult Once a larval has matured we should not change its adult value as this will break those who have already dereferenced it. Signed-off-by: Herbert Xu commit 89a8cd9024a2c3757f6af48e33c8740f66bcd5d4 Author: Herbert Xu Date: Sun Jul 2 00:09:51 2006 +1000 [CRYPTO] digest: Removed old HMAC implementation This patch removes the old HMAC implementation now that nobody uses it anymore. Signed-off-by: Herbert Xu Signed-off-by: David S. Miller commit c6f68733e0cdb6c040a67f168f5eb152d4fbca14 Author: Herbert Xu Date: Sun Jul 2 00:56:35 2006 +1000 [SCTP]: Use HMAC template This patch converts SCTP to use the new HMAC template. Signed-off-by: Herbert Xu Signed-off-by: David S. Miller commit b8d833d8f0da78c5ba61df574916048523ed08ab Author: Herbert Xu Date: Sun Jul 2 00:08:02 2006 +1000 [IPSEC]: Use HMAC template This patch converts IPsec to use the new HMAC template. The names of existing simple digest algorithms may still be used to refer to their HMAC composites. The same structure can be used by other MACs such as AES-XCBC-MAC. Signed-off-by: Herbert Xu Signed-off-by: David S. Miller commit 8f7530d25263fa788447543abc4501369cc12705 Author: Herbert Xu Date: Sat Jul 1 23:56:48 2006 +1000 [CRYPTO] tcrypt: Use HMAC template This patch converts tcrypt to use the new HMAC template rather than the hard-coded version of HMAC. Signed-off-by: Herbert Xu Signed-off-by: David S. Miller commit 2cf3585e23168d73d160520db3a644348741e832 Author: Herbert Xu Date: Mon Jul 3 00:01:18 2006 +1000 [CRYPTO] hmac: Add crypto template implementation This patch rewrites HMAC as a crypto template. This means that HMAC is no longer a hard-coded part of the API. It's now a template that generates standard digest algorithms like any other. The old HMAC is preserved until all current users are converted. The same structure can be used by other MACs such as AES-XCBC-MAC. Signed-off-by: Herbert Xu Signed-off-by: David S. Miller commit 340f0ec9ed36bf70eb80660a120048b68b20e4eb Author: Herbert Xu Date: Sun Jun 25 23:58:09 2006 +1000 [CRYPTO] digest: Added direct SG update and digest interface This patch allows digest algorithms to provide an SG update function as well as a direct digest function. If the SG update function is present, it will be used instead of the normal update interface. That is, the algorithm will be directly responsible for walking the SG list. If the digest function is present, it will take precedence over the generic digest function. This can be used by hardware that is able (and/or only able) to digest in one operation. Signed-off-by: Herbert Xu Signed-off-by: David S. Miller commit 98d692a99bd0f728bb5f570a77a247eff6301f4d Author: Herbert Xu Date: Sun Jul 2 23:45:02 2006 +1000 [CRYPTO] api: Added spawns Spawns lock a specific crypto algorithm in place. They can then be used with crypto_spawn_tfm to allocate a tfm for that algorithm. When the base algorithm of a spawn is deregistered, all its spawns will be automatically removed. Signed-off-by: Herbert Xu Signed-off-by: David S. Miller commit 25305da609f7b9ac3f4133dab884fd15ac7f8ec0 Author: Herbert Xu Date: Sun Jul 2 23:34:29 2006 +1000 [CRYPTO] api: Allow algorithm lookup by type This patch also adds the infrastructure to pick an algorithm based on their type. For example, this allows you to select the encryption algorithm "aes", instead of any algorithm registered under the name "aes". For now this is only accessible internally. Eventually it will be made available through crypto_alloc_tfm. Signed-off-by: Herbert Xu Signed-off-by: David S. Miller commit 44f1f1d83168ca55577143d0593c7d4499318fb1 Author: Herbert Xu Date: Mon Jul 3 00:05:17 2006 +1000 [CRYPTO] api: Add cryptomgr The cryptomgr module is a simple manager of crypto algorithm instances. It ensures that parameterised algorithms of the type tmpl(alg) (e.g., cbc(aes)) are always created. This is meant to satisfy the needs for most users. For more complex cases such as deeper combinations or multiple parameters, a netlink module will be created which allows arbitrary expressions to be parsed in user-space. Signed-off-by: Herbert Xu Signed-off-by: David S. Miller commit 56538c6c510bb90cbaeeb094fcc6f35fe6a2d0c0 Author: Herbert Xu Date: Sun Jul 2 23:33:45 2006 +1000 [CRYPTO] api: Added event notification This patch adds a notifier chain for algorithm/template registration events. This will be used to register compound algorithms such as cbc(aes). In future this will also be passed onto user-space through netlink. Signed-off-by: Herbert Xu Signed-off-by: David S. Miller commit 3c1a00af7c31186bbeaa314f0fdea232c09e2ffa Author: Herbert Xu Date: Mon Jul 3 00:04:43 2006 +1000 [CRYPTO] api: Add template registration A crypto_template generates a crypto_alg object when given a set of parameters. this patch adds the basic data structure fo templates and code to handle their registration/deregistration. Signed-off-by: Herbert Xu Signed-off-by: David S. Miller commit 33699d7e36043e8789ad38218cf6062a9a82df03 Author: Herbert Xu Date: Sun Jul 2 21:32:03 2006 +1000 [CRYPTO] api: Add crypto_alg reference counting Up until now we've relied on module reference counting to ensure that the crypto_alg structures don't disappear from under us. This was good enough as long as each crypto_alg came from exactly one module. However, with parameterised crypto algorithms a crypto_alg object may need two or more modules to operate. This means that we need to count the references to the crypto_alg object directly. Signed-off-by: Herbert Xu Signed-off-by: David S. Miller commit 970ff99ff9637a392290c32ccef9e73f60c2d17a Author: Herbert Xu Date: Sun May 28 09:05:24 2006 +1000 [CRYPTO] api: Rename crypto_alg_get to crypto_mod_get The functions crypto_alg_get and crypto_alg_put operates on the crypto modules rather than the algorithms. Therefore it makes sense to call them crypto_mod_get and crypto_alg_put respectively. This is needed because we need to have real algorithm reference counters for parameterised algorithms as they can be unregistered from below by when their parameter algorithms are themselves unregistered. Signed-off-by: Herbert Xu Signed-off-by: David S. Miller commit ed3a7baae82d63c171a88a98b6abd491d5ded19e Author: Joachim Fritschi Date: Tue Jun 20 21:12:02 2006 +1000 [CRYPTO] twofish: x86-64 assembly version The patch passed the trycpt tests and automated filesystem tests. This rewrite resulted in some nice perfomance increase over my last patch. Short summary of the tcrypt benchmarks: Twofish Assembler vs. Twofish C (256bit 8kb block CBC) encrypt: -27% Cycles decrypt: -23% Cycles Twofish Assembler vs. AES Assembler (128bit 8kb block CBC) encrypt: +18% Cycles decrypt: +15% Cycles Twofish Assembler vs. AES Assembler (256bit 8kb block CBC) encrypt: -9% Cycles decrypt: -8% Cycles Full Output: http://homepages.tu-darmstadt.de/~fritschi/twofish/tcrypt-speed-twofish-c-x86_64.txt http://homepages.tu-darmstadt.de/~fritschi/twofish/tcrypt-speed-twofish-asm-x86_64.txt http://homepages.tu-darmstadt.de/~fritschi/twofish/tcrypt-speed-aes-asm-x86_64.txt Here is another bonnie++ benchmark with encrypted filesystems. Most runs maxed out the hd. It should give some idea what the module can do for encrypted filesystem performance even though you can't see the full numbers. http://homepages.tu-darmstadt.de/~fritschi/twofish/output_20060610_130806_x86_64.html Signed-off-by: Joachim Fritschi Signed-off-by: Herbert Xu commit b2756612b21c8457c8408f81bd1fe9c56c1eae47 Author: Joachim Fritschi Date: Tue Jun 20 20:59:16 2006 +1000 [CRYPTO] twofish: i586 assembly version The patch passed the trycpt tests and automated filesystem tests. This rewrite resulted in some nice perfomance increase over my last patch. Short summary of the tcrypt benchmarks: Twofish Assembler vs. Twofish C (256bit 8kb block CBC) encrypt: -33% Cycles decrypt: -45% Cycles Twofish Assembler vs. AES Assembler (128bit 8kb block CBC) encrypt: +3% Cycles decrypt: -22% Cycles Twofish Assembler vs. AES Assembler (256bit 8kb block CBC) encrypt: -20% Cycles decrypt: -36% Cycles Full Output: http://homepages.tu-darmstadt.de/~fritschi/twofish/tcrypt-speed-twofish-asm-i586.txt http://homepages.tu-darmstadt.de/~fritschi/twofish/tcrypt-speed-twofish-c-i586.txt http://homepages.tu-darmstadt.de/~fritschi/twofish/tcrypt-speed-aes-asm-i586.txt Here is another bonnie++ benchmark with encrypted filesystems. All runs with the twofish assembler modules max out the drivespeed. It should give some idea what the module can do for encrypted filesystem performance even though you can't see the full numbers. http://homepages.tu-darmstadt.de/~fritschi/twofish/output_20060611_205432_x86.html Signed-off-by: Joachim Fritschi Signed-off-by: Herbert Xu commit e55e9f2aba29d62c3021a4849b0eda67fdb9d55e Author: Joachim Fritschi Date: Tue Jun 20 20:39:29 2006 +1000 [CRYPTO] twofish: Fix the priority This patch adds a proper driver name and priority to the generic c implemtation to allow coexistance of c and assembler modules. Signed-off-by: Joachim Fritschi Signed-off-by: Herbert Xu commit 2af6a24ce954a03e4f0de07bd45d9e81d17af3e9 Author: Joachim Fritschi Date: Tue Jun 20 20:37:23 2006 +1000 [CRYPTO] twofish: Split out common c code This patch splits up the twofish crypto routine into a common part ( key setup ) which will be uses by all twofish crypto modules ( generic-c , i586 assembler and x86_64 assembler ) and generic-c part. It also creates a new header file which will be used by all 3 modules. This eliminates all code duplication. Correctness was verified with the tcrypt module and automated test scripts. Signed-off-by: Joachim Fritschi Signed-off-by: Herbert Xu --- diff --git a/arch/i386/crypto/Makefile b/arch/i386/crypto/Makefile index 103c353..3fd19af 100644 --- a/arch/i386/crypto/Makefile +++ b/arch/i386/crypto/Makefile @@ -5,5 +5,8 @@ # Arch-specific CryptoAPI modules. # obj-$(CONFIG_CRYPTO_AES_586) += aes-i586.o +obj-$(CONFIG_CRYPTO_TWOFISH_586) += twofish-i586.o aes-i586-y := aes-i586-asm.o aes.o +twofish-i586-y := twofish-i586-asm.o twofish.o + diff --git a/arch/i386/crypto/aes.c b/arch/i386/crypto/aes.c index d3806da..49aad93 100644 --- a/arch/i386/crypto/aes.c +++ b/arch/i386/crypto/aes.c @@ -379,12 +379,13 @@ #define kdl8(k,i) \ } static int aes_set_key(struct crypto_tfm *tfm, const u8 *in_key, - unsigned int key_len, u32 *flags) + unsigned int key_len) { int i; u32 ss[8]; struct aes_ctx *ctx = crypto_tfm_ctx(tfm); const __le32 *key = (const __le32 *)in_key; + u32 *flags = &tfm->crt_flags; /* encryption schedule */ diff --git a/arch/i386/crypto/twofish-i586-asm.S b/arch/i386/crypto/twofish-i586-asm.S new file mode 100644 index 0000000..39b98ed --- /dev/null +++ b/arch/i386/crypto/twofish-i586-asm.S @@ -0,0 +1,335 @@ +/*************************************************************************** +* Copyright (C) 2006 by Joachim Fritschi, * +* * +* This program is free software; you can redistribute it and/or modify * +* it under the terms of the GNU General Public License as published by * +* the Free Software Foundation; either version 2 of the License, or * +* (at your option) any later version. * +* * +* This program is distributed in the hope that it will be useful, * +* but WITHOUT ANY WARRANTY; without even the implied warranty of * +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the * +* GNU General Public License for more details. * +* * +* You should have received a copy of the GNU General Public License * +* along with this program; if not, write to the * +* Free Software Foundation, Inc., * +* 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. * +***************************************************************************/ + +.file "twofish-i586-asm.S" +.text + +#include + +/* return adress at 0 */ + +#define in_blk 12 /* input byte array address parameter*/ +#define out_blk 8 /* output byte array address parameter*/ +#define tfm 4 /* Twofish context structure */ + +#define a_offset 0 +#define b_offset 4 +#define c_offset 8 +#define d_offset 12 + +/* Structure of the crypto context struct*/ + +#define s0 0 /* S0 Array 256 Words each */ +#define s1 1024 /* S1 Array */ +#define s2 2048 /* S2 Array */ +#define s3 3072 /* S3 Array */ +#define w 4096 /* 8 whitening keys (word) */ +#define k 4128 /* key 1-32 ( word ) */ + +/* define a few register aliases to allow macro substitution */ + +#define R0D %eax +#define R0B %al +#define R0H %ah + +#define R1D %ebx +#define R1B %bl +#define R1H %bh + +#define R2D %ecx +#define R2B %cl +#define R2H %ch + +#define R3D %edx +#define R3B %dl +#define R3H %dh + + +/* performs input whitening */ +#define input_whitening(src,context,offset)\ + xor w+offset(context), src; + +/* performs input whitening */ +#define output_whitening(src,context,offset)\ + xor w+16+offset(context), src; + +/* + * a input register containing a (rotated 16) + * b input register containing b + * c input register containing c + * d input register containing d (already rol $1) + * operations on a and b are interleaved to increase performance + */ +#define encrypt_round(a,b,c,d,round)\ + push d ## D;\ + movzx b ## B, %edi;\ + mov s1(%ebp,%edi,4),d ## D;\ + movzx a ## B, %edi;\ + mov s2(%ebp,%edi,4),%esi;\ + movzx b ## H, %edi;\ + ror $16, b ## D;\ + xor s2(%ebp,%edi,4),d ## D;\ + movzx a ## H, %edi;\ + ror $16, a ## D;\ + xor s3(%ebp,%edi,4),%esi;\ + movzx b ## B, %edi;\ + xor s3(%ebp,%edi,4),d ## D;\ + movzx a ## B, %edi;\ + xor (%ebp,%edi,4), %esi;\ + movzx b ## H, %edi;\ + ror $15, b ## D;\ + xor (%ebp,%edi,4), d ## D;\ + movzx a ## H, %edi;\ + xor s1(%ebp,%edi,4),%esi;\ + pop %edi;\ + add d ## D, %esi;\ + add %esi, d ## D;\ + add k+round(%ebp), %esi;\ + xor %esi, c ## D;\ + rol $15, c ## D;\ + add k+4+round(%ebp),d ## D;\ + xor %edi, d ## D; + +/* + * a input register containing a (rotated 16) + * b input register containing b + * c input register containing c + * d input register containing d (already rol $1) + * operations on a and b are interleaved to increase performance + * last round has different rotations for the output preparation + */ +#define encrypt_last_round(a,b,c,d,round)\ + push d ## D;\ + movzx b ## B, %edi;\ + mov s1(%ebp,%edi,4),d ## D;\ + movzx a ## B, %edi;\ + mov s2(%ebp,%edi,4),%esi;\ + movzx b ## H, %edi;\ + ror $16, b ## D;\ + xor s2(%ebp,%edi,4),d ## D;\ + movzx a ## H, %edi;\ + ror $16, a ## D;\ + xor s3(%ebp,%edi,4),%esi;\ + movzx b ## B, %edi;\ + xor s3(%ebp,%edi,4),d ## D;\ + movzx a ## B, %edi;\ + xor (%ebp,%edi,4), %esi;\ + movzx b ## H, %edi;\ + ror $16, b ## D;\ + xor (%ebp,%edi,4), d ## D;\ + movzx a ## H, %edi;\ + xor s1(%ebp,%edi,4),%esi;\ + pop %edi;\ + add d ## D, %esi;\ + add %esi, d ## D;\ + add k+round(%ebp), %esi;\ + xor %esi, c ## D;\ + ror $1, c ## D;\ + add k+4+round(%ebp),d ## D;\ + xor %edi, d ## D; + +/* + * a input register containing a + * b input register containing b (rotated 16) + * c input register containing c + * d input register containing d (already rol $1) + * operations on a and b are interleaved to increase performance + */ +#define decrypt_round(a,b,c,d,round)\ + push c ## D;\ + movzx a ## B, %edi;\ + mov (%ebp,%edi,4), c ## D;\ + movzx b ## B, %edi;\ + mov s3(%ebp,%edi,4),%esi;\ + movzx a ## H, %edi;\ + ror $16, a ## D;\ + xor s1(%ebp,%edi,4),c ## D;\ + movzx b ## H, %edi;\ + ror $16, b ## D;\ + xor (%ebp,%edi,4), %esi;\ + movzx a ## B, %edi;\ + xor s2(%ebp,%edi,4),c ## D;\ + movzx b ## B, %edi;\ + xor s1(%ebp,%edi,4),%esi;\ + movzx a ## H, %edi;\ + ror $15, a ## D;\ + xor s3(%ebp,%edi,4),c ## D;\ + movzx b ## H, %edi;\ + xor s2(%ebp,%edi,4),%esi;\ + pop %edi;\ + add %esi, c ## D;\ + add c ## D, %esi;\ + add k+round(%ebp), c ## D;\ + xor %edi, c ## D;\ + add k+4+round(%ebp),%esi;\ + xor %esi, d ## D;\ + rol $15, d ## D; + +/* + * a input register containing a + * b input register containing b (rotated 16) + * c input register containing c + * d input register containing d (already rol $1) + * operations on a and b are interleaved to increase performance + * last round has different rotations for the output preparation + */ +#define decrypt_last_round(a,b,c,d,round)\ + push c ## D;\ + movzx a ## B, %edi;\ + mov (%ebp,%edi,4), c ## D;\ + movzx b ## B, %edi;\ + mov s3(%ebp,%edi,4),%esi;\ + movzx a ## H, %edi;\ + ror $16, a ## D;\ + xor s1(%ebp,%edi,4),c ## D;\ + movzx b ## H, %edi;\ + ror $16, b ## D;\ + xor (%ebp,%edi,4), %esi;\ + movzx a ## B, %edi;\ + xor s2(%ebp,%edi,4),c ## D;\ + movzx b ## B, %edi;\ + xor s1(%ebp,%edi,4),%esi;\ + movzx a ## H, %edi;\ + ror $16, a ## D;\ + xor s3(%ebp,%edi,4),c ## D;\ + movzx b ## H, %edi;\ + xor s2(%ebp,%edi,4),%esi;\ + pop %edi;\ + add %esi, c ## D;\ + add c ## D, %esi;\ + add k+round(%ebp), c ## D;\ + xor %edi, c ## D;\ + add k+4+round(%ebp),%esi;\ + xor %esi, d ## D;\ + ror $1, d ## D; + +.align 4 +.global twofish_enc_blk +.global twofish_dec_blk + +twofish_enc_blk: + push %ebp /* save registers according to calling convention*/ + push %ebx + push %esi + push %edi + + mov tfm + 16(%esp), %ebp /* abuse the base pointer: set new base bointer to the crypto tfm */ + add $crypto_tfm_ctx_offset, %ebp /* ctx adress */ + mov in_blk+16(%esp),%edi /* input adress in edi */ + + mov (%edi), %eax + mov b_offset(%edi), %ebx + mov c_offset(%edi), %ecx + mov d_offset(%edi), %edx + input_whitening(%eax,%ebp,a_offset) + ror $16, %eax + input_whitening(%ebx,%ebp,b_offset) + input_whitening(%ecx,%ebp,c_offset) + input_whitening(%edx,%ebp,d_offset) + rol $1, %edx + + encrypt_round(R0,R1,R2,R3,0); + encrypt_round(R2,R3,R0,R1,8); + encrypt_round(R0,R1,R2,R3,2*8); + encrypt_round(R2,R3,R0,R1,3*8); + encrypt_round(R0,R1,R2,R3,4*8); + encrypt_round(R2,R3,R0,R1,5*8); + encrypt_round(R0,R1,R2,R3,6*8); + encrypt_round(R2,R3,R0,R1,7*8); + encrypt_round(R0,R1,R2,R3,8*8); + encrypt_round(R2,R3,R0,R1,9*8); + encrypt_round(R0,R1,R2,R3,10*8); + encrypt_round(R2,R3,R0,R1,11*8); + encrypt_round(R0,R1,R2,R3,12*8); + encrypt_round(R2,R3,R0,R1,13*8); + encrypt_round(R0,R1,R2,R3,14*8); + encrypt_last_round(R2,R3,R0,R1,15*8); + + output_whitening(%eax,%ebp,c_offset) + output_whitening(%ebx,%ebp,d_offset) + output_whitening(%ecx,%ebp,a_offset) + output_whitening(%edx,%ebp,b_offset) + mov out_blk+16(%esp),%edi; + mov %eax, c_offset(%edi) + mov %ebx, d_offset(%edi) + mov %ecx, (%edi) + mov %edx, b_offset(%edi) + + pop %edi + pop %esi + pop %ebx + pop %ebp + mov $1, %eax + ret + +twofish_dec_blk: + push %ebp /* save registers according to calling convention*/ + push %ebx + push %esi + push %edi + + + mov tfm + 16(%esp), %ebp /* abuse the base pointer: set new base bointer to the crypto tfm */ + add $crypto_tfm_ctx_offset, %ebp /* ctx adress */ + mov in_blk+16(%esp),%edi /* input adress in edi */ + + mov (%edi), %eax + mov b_offset(%edi), %ebx + mov c_offset(%edi), %ecx + mov d_offset(%edi), %edx + output_whitening(%eax,%ebp,a_offset) + output_whitening(%ebx,%ebp,b_offset) + ror $16, %ebx + output_whitening(%ecx,%ebp,c_offset) + output_whitening(%edx,%ebp,d_offset) + rol $1, %ecx + + decrypt_round(R0,R1,R2,R3,15*8); + decrypt_round(R2,R3,R0,R1,14*8); + decrypt_round(R0,R1,R2,R3,13*8); + decrypt_round(R2,R3,R0,R1,12*8); + decrypt_round(R0,R1,R2,R3,11*8); + decrypt_round(R2,R3,R0,R1,10*8); + decrypt_round(R0,R1,R2,R3,9*8); + decrypt_round(R2,R3,R0,R1,8*8); + decrypt_round(R0,R1,R2,R3,7*8); + decrypt_round(R2,R3,R0,R1,6*8); + decrypt_round(R0,R1,R2,R3,5*8); + decrypt_round(R2,R3,R0,R1,4*8); + decrypt_round(R0,R1,R2,R3,3*8); + decrypt_round(R2,R3,R0,R1,2*8); + decrypt_round(R0,R1,R2,R3,1*8); + decrypt_last_round(R2,R3,R0,R1,0); + + input_whitening(%eax,%ebp,c_offset) + input_whitening(%ebx,%ebp,d_offset) + input_whitening(%ecx,%ebp,a_offset) + input_whitening(%edx,%ebp,b_offset) + mov out_blk+16(%esp),%edi; + mov %eax, c_offset(%edi) + mov %ebx, d_offset(%edi) + mov %ecx, (%edi) + mov %edx, b_offset(%edi) + + pop %edi + pop %esi + pop %ebx + pop %ebp + mov $1, %eax + ret diff --git a/arch/i386/crypto/twofish.c b/arch/i386/crypto/twofish.c new file mode 100644 index 0000000..e3004df --- /dev/null +++ b/arch/i386/crypto/twofish.c @@ -0,0 +1,97 @@ +/* + * Glue Code for optimized 586 assembler version of TWOFISH + * + * Originally Twofish for GPG + * By Matthew Skala , July 26, 1998 + * 256-bit key length added March 20, 1999 + * Some modifications to reduce the text size by Werner Koch, April, 1998 + * Ported to the kerneli patch by Marc Mutz + * Ported to CryptoAPI by Colin Slater + * + * The original author has disclaimed all copyright interest in this + * code and thus put it in the public domain. The subsequent authors + * have put this under the GNU General Public License. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 + * USA + * + * This code is a "clean room" implementation, written from the paper + * _Twofish: A 128-Bit Block Cipher_ by Bruce Schneier, John Kelsey, + * Doug Whiting, David Wagner, Chris Hall, and Niels Ferguson, available + * through http://www.counterpane.com/twofish.html + * + * For background information on multiplication in finite fields, used for + * the matrix operations in the key schedule, see the book _Contemporary + * Abstract Algebra_ by Joseph A. Gallian, especially chapter 22 in the + * Third Edition. + */ + +#include +#include +#include +#include +#include + + +asmlinkage void twofish_enc_blk(struct crypto_tfm *tfm, u8 *dst, const u8 *src); +asmlinkage void twofish_dec_blk(struct crypto_tfm *tfm, u8 *dst, const u8 *src); + +static void twofish_encrypt(struct crypto_tfm *tfm, u8 *dst, const u8 *src) +{ + twofish_enc_blk(tfm, dst, src); +} + +static void twofish_decrypt(struct crypto_tfm *tfm, u8 *dst, const u8 *src) +{ + twofish_dec_blk(tfm, dst, src); +} + +static struct crypto_alg alg = { + .cra_name = "twofish", + .cra_driver_name = "twofish-i586", + .cra_priority = 200, + .cra_flags = CRYPTO_ALG_TYPE_CIPHER, + .cra_blocksize = TF_BLOCK_SIZE, + .cra_ctxsize = sizeof(struct twofish_ctx), + .cra_alignmask = 3, + .cra_module = THIS_MODULE, + .cra_list = LIST_HEAD_INIT(alg.cra_list), + .cra_u = { + .cipher = { + .cia_min_keysize = TF_MIN_KEY_SIZE, + .cia_max_keysize = TF_MAX_KEY_SIZE, + .cia_setkey = twofish_setkey, + .cia_encrypt = twofish_encrypt, + .cia_decrypt = twofish_decrypt + } + } +}; + +static int __init init(void) +{ + return crypto_register_alg(&alg); +} + +static void __exit fini(void) +{ + crypto_unregister_alg(&alg); +} + +module_init(init); +module_exit(fini); + +MODULE_LICENSE("GPL"); +MODULE_DESCRIPTION ("Twofish Cipher Algorithm, i586 asm optimized"); +MODULE_ALIAS("twofish"); diff --git a/arch/s390/crypto/aes_s390.c b/arch/s390/crypto/aes_s390.c index 5713c7e..c7c43c9 100644 --- a/arch/s390/crypto/aes_s390.c +++ b/arch/s390/crypto/aes_s390.c @@ -38,9 +38,10 @@ struct s390_aes_ctx { }; static int aes_set_key(struct crypto_tfm *tfm, const u8 *in_key, - unsigned int key_len, u32 *flags) + unsigned int key_len) { struct s390_aes_ctx *sctx = crypto_tfm_ctx(tfm); + u32 *flags = &tfm->crt_flags; switch (key_len) { case 16: diff --git a/arch/s390/crypto/des_s390.c b/arch/s390/crypto/des_s390.c index b3f7496..170757b 100644 --- a/arch/s390/crypto/des_s390.c +++ b/arch/s390/crypto/des_s390.c @@ -45,9 +45,10 @@ struct crypt_s390_des3_192_ctx { }; static int des_setkey(struct crypto_tfm *tfm, const u8 *key, - unsigned int keylen, u32 *flags) + unsigned int keylen) { struct crypt_s390_des_ctx *dctx = crypto_tfm_ctx(tfm); + u32 *flags = &tfm->crt_flags; int ret; /* test if key is valid (not a weak key) */ @@ -167,11 +168,12 @@ static struct crypto_alg des_alg = { * */ static int des3_128_setkey(struct crypto_tfm *tfm, const u8 *key, - unsigned int keylen, u32 *flags) + unsigned int keylen) { int i, ret; struct crypt_s390_des3_128_ctx *dctx = crypto_tfm_ctx(tfm); - const u8* temp_key = key; + const u8 *temp_key = key; + u32 *flags = &tfm->crt_flags; if (!(memcmp(key, &key[DES_KEY_SIZE], DES_KEY_SIZE))) { *flags |= CRYPTO_TFM_RES_BAD_KEY_SCHED; @@ -303,11 +305,12 @@ static struct crypto_alg des3_128_alg = * */ static int des3_192_setkey(struct crypto_tfm *tfm, const u8 *key, - unsigned int keylen, u32 *flags) + unsigned int keylen) { int i, ret; struct crypt_s390_des3_192_ctx *dctx = crypto_tfm_ctx(tfm); - const u8* temp_key = key; + const u8 *temp_key = key; + u32 *flags = &tfm->crt_flags; if (!(memcmp(key, &key[DES_KEY_SIZE], DES_KEY_SIZE) && memcmp(&key[DES_KEY_SIZE], &key[DES_KEY_SIZE * 2], diff --git a/arch/x86_64/crypto/Makefile b/arch/x86_64/crypto/Makefile index 426d20f..15b538a 100644 --- a/arch/x86_64/crypto/Makefile +++ b/arch/x86_64/crypto/Makefile @@ -5,5 +5,8 @@ # Arch-specific CryptoAPI modules. # obj-$(CONFIG_CRYPTO_AES_X86_64) += aes-x86_64.o +obj-$(CONFIG_CRYPTO_TWOFISH_X86_64) += twofish-x86_64.o aes-x86_64-y := aes-x86_64-asm.o aes.o +twofish-x86_64-y := twofish-x86_64-asm.o twofish.o + diff --git a/arch/x86_64/crypto/aes.c b/arch/x86_64/crypto/aes.c index 68866fa..5cdb13e 100644 --- a/arch/x86_64/crypto/aes.c +++ b/arch/x86_64/crypto/aes.c @@ -228,13 +228,14 @@ #define loop8(i) \ } static int aes_set_key(struct crypto_tfm *tfm, const u8 *in_key, - unsigned int key_len, u32 *flags) + unsigned int key_len) { struct aes_ctx *ctx = crypto_tfm_ctx(tfm); const __le32 *key = (const __le32 *)in_key; + u32 *flags = &tfm->crt_flags; u32 i, j, t, u, v, w; - if (key_len != 16 && key_len != 24 && key_len != 32) { + if (key_len % 8) { *flags |= CRYPTO_TFM_RES_BAD_KEY_LEN; return -EINVAL; } diff --git a/arch/x86_64/crypto/twofish-x86_64-asm.S b/arch/x86_64/crypto/twofish-x86_64-asm.S new file mode 100644 index 0000000..35974a5 --- /dev/null +++ b/arch/x86_64/crypto/twofish-x86_64-asm.S @@ -0,0 +1,324 @@ +/*************************************************************************** +* Copyright (C) 2006 by Joachim Fritschi, * +* * +* This program is free software; you can redistribute it and/or modify * +* it under the terms of the GNU General Public License as published by * +* the Free Software Foundation; either version 2 of the License, or * +* (at your option) any later version. * +* * +* This program is distributed in the hope that it will be useful, * +* but WITHOUT ANY WARRANTY; without even the implied warranty of * +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the * +* GNU General Public License for more details. * +* * +* You should have received a copy of the GNU General Public License * +* along with this program; if not, write to the * +* Free Software Foundation, Inc., * +* 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. * +***************************************************************************/ + +.file "twofish-x86_64-asm.S" +.text + +#include + +#define a_offset 0 +#define b_offset 4 +#define c_offset 8 +#define d_offset 12 + +/* Structure of the crypto context struct*/ + +#define s0 0 /* S0 Array 256 Words each */ +#define s1 1024 /* S1 Array */ +#define s2 2048 /* S2 Array */ +#define s3 3072 /* S3 Array */ +#define w 4096 /* 8 whitening keys (word) */ +#define k 4128 /* key 1-32 ( word ) */ + +/* define a few register aliases to allow macro substitution */ + +#define R0 %rax +#define R0D %eax +#define R0B %al +#define R0H %ah + +#define R1 %rbx +#define R1D %ebx +#define R1B %bl +#define R1H %bh + +#define R2 %rcx +#define R2D %ecx +#define R2B %cl +#define R2H %ch + +#define R3 %rdx +#define R3D %edx +#define R3B %dl +#define R3H %dh + + +/* performs input whitening */ +#define input_whitening(src,context,offset)\ + xor w+offset(context), src; + +/* performs input whitening */ +#define output_whitening(src,context,offset)\ + xor w+16+offset(context), src; + + +/* + * a input register containing a (rotated 16) + * b input register containing b + * c input register containing c + * d input register containing d (already rol $1) + * operations on a and b are interleaved to increase performance + */ +#define encrypt_round(a,b,c,d,round)\ + movzx b ## B, %edi;\ + mov s1(%r11,%rdi,4),%r8d;\ + movzx a ## B, %edi;\ + mov s2(%r11,%rdi,4),%r9d;\ + movzx b ## H, %edi;\ + ror $16, b ## D;\ + xor s2(%r11,%rdi,4),%r8d;\ + movzx a ## H, %edi;\ + ror $16, a ## D;\ + xor s3(%r11,%rdi,4),%r9d;\ + movzx b ## B, %edi;\ + xor s3(%r11,%rdi,4),%r8d;\ + movzx a ## B, %edi;\ + xor (%r11,%rdi,4), %r9d;\ + movzx b ## H, %edi;\ + ror $15, b ## D;\ + xor (%r11,%rdi,4), %r8d;\ + movzx a ## H, %edi;\ + xor s1(%r11,%rdi,4),%r9d;\ + add %r8d, %r9d;\ + add %r9d, %r8d;\ + add k+round(%r11), %r9d;\ + xor %r9d, c ## D;\ + rol $15, c ## D;\ + add k+4+round(%r11),%r8d;\ + xor %r8d, d ## D; + +/* + * a input register containing a(rotated 16) + * b input register containing b + * c input register containing c + * d input register containing d (already rol $1) + * operations on a and b are interleaved to increase performance + * during the round a and b are prepared for the output whitening + */ +#define encrypt_last_round(a,b,c,d,round)\ + mov b ## D, %r10d;\ + shl $32, %r10;\ + movzx b ## B, %edi;\ + mov s1(%r11,%rdi,4),%r8d;\ + movzx a ## B, %edi;\ + mov s2(%r11,%rdi,4),%r9d;\ + movzx b ## H, %edi;\ + ror $16, b ## D;\ + xor s2(%r11,%rdi,4),%r8d;\ + movzx a ## H, %edi;\ + ror $16, a ## D;\ + xor s3(%r11,%rdi,4),%r9d;\ + movzx b ## B, %edi;\ + xor s3(%r11,%rdi,4),%r8d;\ + movzx a ## B, %edi;\ + xor (%r11,%rdi,4), %r9d;\ + xor a, %r10;\ + movzx b ## H, %edi;\ + xor (%r11,%rdi,4), %r8d;\ + movzx a ## H, %edi;\ + xor s1(%r11,%rdi,4),%r9d;\ + add %r8d, %r9d;\ + add %r9d, %r8d;\ + add k+round(%r11), %r9d;\ + xor %r9d, c ## D;\ + ror $1, c ## D;\ + add k+4+round(%r11),%r8d;\ + xor %r8d, d ## D + +/* + * a input register containing a + * b input register containing b (rotated 16) + * c input register containing c (already rol $1) + * d input register containing d + * operations on a and b are interleaved to increase performance + */ +#define decrypt_round(a,b,c,d,round)\ + movzx a ## B, %edi;\ + mov (%r11,%rdi,4), %r9d;\ + movzx b ## B, %edi;\ + mov s3(%r11,%rdi,4),%r8d;\ + movzx a ## H, %edi;\ + ror $16, a ## D;\ + xor s1(%r11,%rdi,4),%r9d;\ + movzx b ## H, %edi;\ + ror $16, b ## D;\ + xor (%r11,%rdi,4), %r8d;\ + movzx a ## B, %edi;\ + xor s2(%r11,%rdi,4),%r9d;\ + movzx b ## B, %edi;\ + xor s1(%r11,%rdi,4),%r8d;\ + movzx a ## H, %edi;\ + ror $15, a ## D;\ + xor s3(%r11,%rdi,4),%r9d;\ + movzx b ## H, %edi;\ + xor s2(%r11,%rdi,4),%r8d;\ + add %r8d, %r9d;\ + add %r9d, %r8d;\ + add k+round(%r11), %r9d;\ + xor %r9d, c ## D;\ + add k+4+round(%r11),%r8d;\ + xor %r8d, d ## D;\ + rol $15, d ## D; + +/* + * a input register containing a + * b input register containing b + * c input register containing c (already rol $1) + * d input register containing d + * operations on a and b are interleaved to increase performance + * during the round a and b are prepared for the output whitening + */ +#define decrypt_last_round(a,b,c,d,round)\ + movzx a ## B, %edi;\ + mov (%r11,%rdi,4), %r9d;\ + movzx b ## B, %edi;\ + mov s3(%r11,%rdi,4),%r8d;\ + movzx b ## H, %edi;\ + ror $16, b ## D;\ + xor (%r11,%rdi,4), %r8d;\ + movzx a ## H, %edi;\ + mov b ## D, %r10d;\ + shl $32, %r10;\ + xor a, %r10;\ + ror $16, a ## D;\ + xor s1(%r11,%rdi,4),%r9d;\ + movzx b ## B, %edi;\ + xor s1(%r11,%rdi,4),%r8d;\ + movzx a ## B, %edi;\ + xor s2(%r11,%rdi,4),%r9d;\ + movzx b ## H, %edi;\ + xor s2(%r11,%rdi,4),%r8d;\ + movzx a ## H, %edi;\ + xor s3(%r11,%rdi,4),%r9d;\ + add %r8d, %r9d;\ + add %r9d, %r8d;\ + add k+round(%r11), %r9d;\ + xor %r9d, c ## D;\ + add k+4+round(%r11),%r8d;\ + xor %r8d, d ## D;\ + ror $1, d ## D; + +.align 8 +.global twofish_enc_blk +.global twofish_dec_blk + +twofish_enc_blk: + pushq R1 + + /* %rdi contains the crypto tfm adress */ + /* %rsi contains the output adress */ + /* %rdx contains the input adress */ + add $crypto_tfm_ctx_offset, %rdi /* set ctx adress */ + /* ctx adress is moved to free one non-rex register + as target for the 8bit high operations */ + mov %rdi, %r11 + + movq (R3), R1 + movq 8(R3), R3 + input_whitening(R1,%r11,a_offset) + input_whitening(R3,%r11,c_offset) + mov R1D, R0D + rol $16, R0D + shr $32, R1 + mov R3D, R2D + shr $32, R3 + rol $1, R3D + + encrypt_round(R0,R1,R2,R3,0); + encrypt_round(R2,R3,R0,R1,8); + encrypt_round(R0,R1,R2,R3,2*8); + encrypt_round(R2,R3,R0,R1,3*8); + encrypt_round(R0,R1,R2,R3,4*8); + encrypt_round(R2,R3,R0,R1,5*8); + encrypt_round(R0,R1,R2,R3,6*8); + encrypt_round(R2,R3,R0,R1,7*8); + encrypt_round(R0,R1,R2,R3,8*8); + encrypt_round(R2,R3,R0,R1,9*8); + encrypt_round(R0,R1,R2,R3,10*8); + encrypt_round(R2,R3,R0,R1,11*8); + encrypt_round(R0,R1,R2,R3,12*8); + encrypt_round(R2,R3,R0,R1,13*8); + encrypt_round(R0,R1,R2,R3,14*8); + encrypt_last_round(R2,R3,R0,R1,15*8); + + + output_whitening(%r10,%r11,a_offset) + movq %r10, (%rsi) + + shl $32, R1 + xor R0, R1 + + output_whitening(R1,%r11,c_offset) + movq R1, 8(%rsi) + + popq R1 + movq $1,%rax + ret + +twofish_dec_blk: + pushq R1 + + /* %rdi contains the crypto tfm adress */ + /* %rsi contains the output adress */ + /* %rdx contains the input adress */ + add $crypto_tfm_ctx_offset, %rdi /* set ctx adress */ + /* ctx adress is moved to free one non-rex register + as target for the 8bit high operations */ + mov %rdi, %r11 + + movq (R3), R1 + movq 8(R3), R3 + output_whitening(R1,%r11,a_offset) + output_whitening(R3,%r11,c_offset) + mov R1D, R0D + shr $32, R1 + rol $16, R1D + mov R3D, R2D + shr $32, R3 + rol $1, R2D + + decrypt_round(R0,R1,R2,R3,15*8); + decrypt_round(R2,R3,R0,R1,14*8); + decrypt_round(R0,R1,R2,R3,13*8); + decrypt_round(R2,R3,R0,R1,12*8); + decrypt_round(R0,R1,R2,R3,11*8); + decrypt_round(R2,R3,R0,R1,10*8); + decrypt_round(R0,R1,R2,R3,9*8); + decrypt_round(R2,R3,R0,R1,8*8); + decrypt_round(R0,R1,R2,R3,7*8); + decrypt_round(R2,R3,R0,R1,6*8); + decrypt_round(R0,R1,R2,R3,5*8); + decrypt_round(R2,R3,R0,R1,4*8); + decrypt_round(R0,R1,R2,R3,3*8); + decrypt_round(R2,R3,R0,R1,2*8); + decrypt_round(R0,R1,R2,R3,1*8); + decrypt_last_round(R2,R3,R0,R1,0); + + input_whitening(%r10,%r11,a_offset) + movq %r10, (%rsi) + + shl $32, R1 + xor R0, R1 + + input_whitening(R1,%r11,c_offset) + movq R1, 8(%rsi) + + popq R1 + movq $1,%rax + ret diff --git a/arch/x86_64/crypto/twofish.c b/arch/x86_64/crypto/twofish.c new file mode 100644 index 0000000..182d91d --- /dev/null +++ b/arch/x86_64/crypto/twofish.c @@ -0,0 +1,97 @@ +/* + * Glue Code for optimized x86_64 assembler version of TWOFISH + * + * Originally Twofish for GPG + * By Matthew Skala , July 26, 1998 + * 256-bit key length added March 20, 1999 + * Some modifications to reduce the text size by Werner Koch, April, 1998 + * Ported to the kerneli patch by Marc Mutz + * Ported to CryptoAPI by Colin Slater + * + * The original author has disclaimed all copyright interest in this + * code and thus put it in the public domain. The subsequent authors + * have put this under the GNU General Public License. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 + * USA + * + * This code is a "clean room" implementation, written from the paper + * _Twofish: A 128-Bit Block Cipher_ by Bruce Schneier, John Kelsey, + * Doug Whiting, David Wagner, Chris Hall, and Niels Ferguson, available + * through http://www.counterpane.com/twofish.html + * + * For background information on multiplication in finite fields, used for + * the matrix operations in the key schedule, see the book _Contemporary + * Abstract Algebra_ by Joseph A. Gallian, especially chapter 22 in the + * Third Edition. + */ + +#include +#include +#include +#include +#include +#include + +asmlinkage void twofish_enc_blk(struct crypto_tfm *tfm, u8 *dst, const u8 *src); +asmlinkage void twofish_dec_blk(struct crypto_tfm *tfm, u8 *dst, const u8 *src); + +static void twofish_encrypt(struct crypto_tfm *tfm, u8 *dst, const u8 *src) +{ + twofish_enc_blk(tfm, dst, src); +} + +static void twofish_decrypt(struct crypto_tfm *tfm, u8 *dst, const u8 *src) +{ + twofish_dec_blk(tfm, dst, src); +} + +static struct crypto_alg alg = { + .cra_name = "twofish", + .cra_driver_name = "twofish-x86_64", + .cra_priority = 200, + .cra_flags = CRYPTO_ALG_TYPE_CIPHER, + .cra_blocksize = TF_BLOCK_SIZE, + .cra_ctxsize = sizeof(struct twofish_ctx), + .cra_alignmask = 3, + .cra_module = THIS_MODULE, + .cra_list = LIST_HEAD_INIT(alg.cra_list), + .cra_u = { + .cipher = { + .cia_min_keysize = TF_MIN_KEY_SIZE, + .cia_max_keysize = TF_MAX_KEY_SIZE, + .cia_setkey = twofish_setkey, + .cia_encrypt = twofish_encrypt, + .cia_decrypt = twofish_decrypt + } + } +}; + +static int __init init(void) +{ + return crypto_register_alg(&alg); +} + +static void __exit fini(void) +{ + crypto_unregister_alg(&alg); +} + +module_init(init); +module_exit(fini); + +MODULE_LICENSE("GPL"); +MODULE_DESCRIPTION ("Twofish Cipher Algorithm, x86_64 asm optimized"); +MODULE_ALIAS("twofish"); diff --git a/crypto/Kconfig b/crypto/Kconfig index ba133d5..00fc4e6 100644 --- a/crypto/Kconfig +++ b/crypto/Kconfig @@ -9,8 +9,16 @@ config CRYPTO help This option provides the core Cryptographic API. +config CRYPTO_MANAGER + tristate "Cryptographic algorithm manager" + depends on CRYPTO + default m + help + Create default cryptographic template instantiations such as + cbc(aes). + config CRYPTO_HMAC - bool "HMAC support" + tristate "HMAC support" depends on CRYPTO help HMAC: Keyed-Hashing for Message Authentication (RFC2104). @@ -131,6 +139,7 @@ config CRYPTO_BLOWFISH config CRYPTO_TWOFISH tristate "Twofish cipher algorithm" depends on CRYPTO + select CRYPTO_TWOFISH_COMMON help Twofish cipher algorithm. @@ -142,6 +151,43 @@ config CRYPTO_TWOFISH See also: +config CRYPTO_TWOFISH_COMMON + tristate + depends on CRYPTO + help + Common parts of the Twofish cipher algorithm shared by the + generic c and the assembler implementations. + +config CRYPTO_TWOFISH_586 + tristate "Twofish cipher algorithms (i586)" + depends on CRYPTO && ((X86 || UML_X86) && !64BIT) + select CRYPTO_TWOFISH_COMMON + help + Twofish cipher algorithm. + + Twofish was submitted as an AES (Advanced Encryption Standard) + candidate cipher by researchers at CounterPane Systems. It is a + 16 round block cipher supporting key sizes of 128, 192, and 256 + bits. + + See also: + + +config CRYPTO_TWOFISH_X86_64 + tristate "Twofish cipher algorithm (x86_64)" + depends on CRYPTO && ((X86 || UML_X86) && 64BIT) + select CRYPTO_TWOFISH_COMMON + help + Twofish cipher algorithm (x86_64). + + Twofish was submitted as an AES (Advanced Encryption Standard) + candidate cipher by researchers at CounterPane Systems. It is a + 16 round block cipher supporting key sizes of 128, 192, and 256 + bits. + + See also: + + config CRYPTO_SERPENT tristate "Serpent cipher algorithm" depends on CRYPTO diff --git a/crypto/Makefile b/crypto/Makefile index d287b9e..2a0183c 100644 --- a/crypto/Makefile +++ b/crypto/Makefile @@ -7,6 +7,7 @@ proc-crypto-$(CONFIG_PROC_FS) = proc.o obj-$(CONFIG_CRYPTO) += api.o scatterwalk.o cipher.o digest.o compress.o \ $(proc-crypto-y) +obj-$(CONFIG_CRYPTO_MANAGER) += cryptomgr.o obj-$(CONFIG_CRYPTO_HMAC) += hmac.o obj-$(CONFIG_CRYPTO_NULL) += crypto_null.o obj-$(CONFIG_CRYPTO_MD4) += md4.o @@ -19,6 +20,7 @@ obj-$(CONFIG_CRYPTO_TGR192) += tgr192.o obj-$(CONFIG_CRYPTO_DES) += des.o obj-$(CONFIG_CRYPTO_BLOWFISH) += blowfish.o obj-$(CONFIG_CRYPTO_TWOFISH) += twofish.o +obj-$(CONFIG_CRYPTO_TWOFISH_COMMON) += twofish_common.o obj-$(CONFIG_CRYPTO_SERPENT) += serpent.o obj-$(CONFIG_CRYPTO_AES) += aes.o obj-$(CONFIG_CRYPTO_CAST5) += cast5.o diff --git a/crypto/aes.c b/crypto/aes.c index a038711..e244077 100644 --- a/crypto/aes.c +++ b/crypto/aes.c @@ -249,13 +249,14 @@ #define loop8(i) } static int aes_set_key(struct crypto_tfm *tfm, const u8 *in_key, - unsigned int key_len, u32 *flags) + unsigned int key_len) { struct aes_ctx *ctx = crypto_tfm_ctx(tfm); const __le32 *key = (const __le32 *)in_key; + u32 *flags = &tfm->crt_flags; u32 i, t, u, v, w; - if (key_len != 16 && key_len != 24 && key_len != 32) { + if (key_len % 8) { *flags |= CRYPTO_TFM_RES_BAD_KEY_LEN; return -EINVAL; } diff --git a/crypto/anubis.c b/crypto/anubis.c index 7e2e1a2..1c771f7 100644 --- a/crypto/anubis.c +++ b/crypto/anubis.c @@ -461,10 +461,11 @@ static const u32 rc[] = { }; static int anubis_setkey(struct crypto_tfm *tfm, const u8 *in_key, - unsigned int key_len, u32 *flags) + unsigned int key_len) { struct anubis_ctx *ctx = crypto_tfm_ctx(tfm); const __be32 *key = (const __be32 *)in_key; + u32 *flags = &tfm->crt_flags; int N, R, i, r; u32 kappa[ANUBIS_MAX_N]; u32 inter[ANUBIS_MAX_N]; diff --git a/crypto/api.c b/crypto/api.c index c11ec1f..23dd53d 100644 --- a/crypto/api.c +++ b/crypto/api.c @@ -18,67 +18,248 @@ #include #include #include +#include #include #include #include +#include +#include +#include #include +#include #include #include #include "internal.h" LIST_HEAD(crypto_alg_list); +static LIST_HEAD(crypto_template_list); DECLARE_RWSEM(crypto_alg_sem); -static inline int crypto_alg_get(struct crypto_alg *alg) +static BLOCKING_NOTIFIER_HEAD(crypto_chain); + +static void crypto_destroy_instance(struct crypto_alg *alg); +static void crypto_remove_final(struct list_head *list); + +static inline struct crypto_alg *crypto_alg_get(struct crypto_alg *alg) { - return try_module_get(alg->cra_module); + atomic_inc(&alg->cra_refcnt); + return alg; } static inline void crypto_alg_put(struct crypto_alg *alg) { + if (atomic_dec_and_test(&alg->cra_refcnt) && alg->cra_destroy) + alg->cra_destroy(alg); +} + +static struct crypto_alg *crypto_mod_get(struct crypto_alg *alg) +{ + return try_module_get(alg->cra_module) ? crypto_alg_get(alg) : NULL; +} + +void crypto_mod_put(struct crypto_alg *alg) +{ + crypto_alg_put(alg); module_put(alg->cra_module); } +EXPORT_SYMBOL_GPL(crypto_mod_put); + +static inline int crypto_is_larval(struct crypto_alg *alg) +{ + return alg->cra_flags & CRYPTO_ALG_LARVAL; +} + +static inline int crypto_is_dead(struct crypto_alg *alg) +{ + return alg->cra_flags & CRYPTO_ALG_DEAD; +} + +static inline int crypto_is_moribund(struct crypto_alg *alg) +{ + return alg->cra_flags & (CRYPTO_ALG_DEAD | CRYPTO_ALG_DYING); +} -static struct crypto_alg *crypto_alg_lookup(const char *name) +static inline int crypto_notify(unsigned long val, void *v) +{ + return blocking_notifier_call_chain(&crypto_chain, val, v); +} + +static struct crypto_alg *__crypto_alg_lookup(const char *name, u32 type, + u32 mask) { struct crypto_alg *q, *alg = NULL; - int best = -1; + int best = -2; - if (!name) - return NULL; - - down_read(&crypto_alg_sem); - list_for_each_entry(q, &crypto_alg_list, cra_list) { int exact, fuzzy; + if (crypto_is_moribund(q)) + continue; + + if ((q->cra_flags ^ type) & mask) + continue; + + if (crypto_is_larval(q) && + ((struct crypto_larval *)q)->mask != mask) + continue; + exact = !strcmp(q->cra_driver_name, name); fuzzy = !strcmp(q->cra_name, name); if (!exact && !(fuzzy && q->cra_priority > best)) continue; - if (unlikely(!crypto_alg_get(q))) + if (unlikely(!crypto_mod_get(q))) continue; best = q->cra_priority; if (alg) - crypto_alg_put(alg); + crypto_mod_put(alg); alg = q; if (exact) break; } - + + return alg; +} + +static void crypto_larval_destroy(struct crypto_alg *alg) +{ + struct crypto_larval *larval = (void *)alg; + + BUG_ON(!crypto_is_larval(alg)); + if (larval->adult) + crypto_mod_put(larval->adult); + kfree(larval); +} + +static struct crypto_alg *crypto_larval_alloc(const char *name, u32 type, + u32 mask) +{ + struct crypto_alg *alg; + struct crypto_larval *larval; + + larval = kzalloc(sizeof(*larval), GFP_KERNEL); + if (!larval) + return ERR_PTR(-ENOMEM); + + larval->mask = mask; + larval->alg.cra_flags = CRYPTO_ALG_LARVAL | type; + larval->alg.cra_priority = -1; + larval->alg.cra_destroy = crypto_larval_destroy; + + atomic_set(&larval->alg.cra_refcnt, 2); + strlcpy(larval->alg.cra_name, name, CRYPTO_MAX_ALG_NAME); + init_completion(&larval->completion); + + down_write(&crypto_alg_sem); + alg = __crypto_alg_lookup(name, type, mask); + if (!alg) { + alg = &larval->alg; + list_add(&alg->cra_list, &crypto_alg_list); + } + up_write(&crypto_alg_sem); + + if (alg != &larval->alg) + kfree(larval); + + return alg; +} + +static void crypto_larval_kill(struct crypto_alg *alg) +{ + struct crypto_larval *larval = (void *)alg; + + down_write(&crypto_alg_sem); + list_del(&alg->cra_list); + up_write(&crypto_alg_sem); + complete(&larval->completion); + crypto_alg_put(alg); +} + +static struct crypto_alg *crypto_larval_wait(struct crypto_alg *alg) +{ + struct crypto_larval *larval = (void *)alg; + + wait_for_completion_interruptible_timeout(&larval->completion, 60 * HZ); + alg = larval->adult; + if (alg) { + if (!crypto_mod_get(alg)) + alg = ERR_PTR(-EAGAIN); + } else + alg = ERR_PTR(-ENOENT); + crypto_mod_put(&larval->alg); + + return alg; +} + +void crypto_larval_error(const char *name, u32 type, u32 mask) +{ + struct crypto_alg *alg; + + down_read(&crypto_alg_sem); + alg = __crypto_alg_lookup(name, type, mask); up_read(&crypto_alg_sem); + + if (alg) { + if (crypto_is_larval(alg)) { + struct crypto_larval *larval = (void *)alg; + complete(&larval->completion); + } + crypto_mod_put(alg); + } +} +EXPORT_SYMBOL_GPL(crypto_larval_error); + +static struct crypto_alg *crypto_alg_lookup(const char *name, u32 type, + u32 mask) +{ + struct crypto_alg *alg; + + down_read(&crypto_alg_sem); + alg = __crypto_alg_lookup(name, type, mask); + up_read(&crypto_alg_sem); + return alg; } -/* A far more intelligent version of this is planned. For now, just - * try an exact match on the name of the algorithm. */ -static inline struct crypto_alg *crypto_alg_mod_lookup(const char *name) +struct crypto_alg *crypto_alg_mod_lookup(const char *name, u32 type, u32 mask) { - return try_then_request_module(crypto_alg_lookup(name), name); + struct crypto_alg *alg; + struct crypto_alg *larval; + int ok; + + if (!name) + return ERR_PTR(-ENOENT); + + mask &= ~(CRYPTO_ALG_LARVAL | CRYPTO_ALG_DEAD); + type &= mask; + + alg = try_then_request_module(crypto_alg_lookup(name, type, mask), + name); + if (alg) + return crypto_is_larval(alg) ? crypto_larval_wait(alg) : alg; + + larval = crypto_larval_alloc(name, type, mask); + if (IS_ERR(larval) || !crypto_is_larval(larval)) + return larval; + + ok = crypto_notify(CRYPTO_MSG_ALG_REQUEST, larval); + if (ok == NOTIFY_DONE) { + request_module("cryptomgr"); + ok = crypto_notify(CRYPTO_MSG_ALG_REQUEST, larval); + } + + if (ok == NOTIFY_STOP) + alg = crypto_larval_wait(larval); + else { + crypto_mod_put(larval); + alg = ERR_PTR(-ENOENT); + } + crypto_larval_kill(larval); + return alg; } +EXPORT_SYMBOL_GPL(crypto_alg_mod_lookup); static int crypto_init_flags(struct crypto_tfm *tfm, u32 flags) { @@ -168,31 +349,39 @@ static unsigned int crypto_ctxsize(struc return len + (alg->cra_alignmask & ~(crypto_tfm_ctx_alignment() - 1)); } -struct crypto_tfm *crypto_alloc_tfm(const char *name, u32 flags) +static void crypto_shoot_alg(struct crypto_alg *alg) +{ + down_write(&crypto_alg_sem); + alg->cra_flags |= CRYPTO_ALG_DYING; + up_write(&crypto_alg_sem); +} + +static struct crypto_tfm *__crypto_alloc_tfm(struct crypto_alg *alg, u32 flags) { struct crypto_tfm *tfm = NULL; - struct crypto_alg *alg; unsigned int tfm_size; - - alg = crypto_alg_mod_lookup(name); - if (alg == NULL) - goto out; + int err = -ENOMEM; tfm_size = sizeof(*tfm) + crypto_ctxsize(alg, flags); tfm = kzalloc(tfm_size, GFP_KERNEL); if (tfm == NULL) - goto out_put; + goto out; tfm->__crt_alg = alg; - - if (crypto_init_flags(tfm, flags)) + + err = crypto_init_flags(tfm, flags); + if (err) goto out_free_tfm; - if (crypto_init_ops(tfm)) + err = crypto_init_ops(tfm); + if (err) goto out_free_tfm; - if (alg->cra_init && alg->cra_init(tfm)) + if (alg->cra_init && (err = alg->cra_init(tfm))) { + if (err == -EAGAIN) + crypto_shoot_alg(alg); goto cra_init_failed; + } goto out; @@ -200,13 +389,36 @@ cra_init_failed: crypto_exit_ops(tfm); out_free_tfm: kfree(tfm); - tfm = NULL; -out_put: - crypto_alg_put(alg); + tfm = ERR_PTR(err); out: return tfm; } +struct crypto_tfm *crypto_alloc_tfm(const char *name, u32 flags) +{ + struct crypto_tfm *tfm = NULL; + int err; + + do { + struct crypto_alg *alg; + + alg = crypto_alg_mod_lookup(name, 0, 0); + err = PTR_ERR(alg); + if (IS_ERR(alg)) + continue; + + tfm = __crypto_alloc_tfm(alg, flags); + err = 0; + if (IS_ERR(tfm)) { + crypto_mod_put(alg); + err = PTR_ERR(tfm); + tfm = NULL; + } + } while (err == -EAGAIN && !signal_pending(current)); + + return tfm; +} + void crypto_free_tfm(struct crypto_tfm *tfm) { struct crypto_alg *alg; @@ -221,7 +433,7 @@ void crypto_free_tfm(struct crypto_tfm * if (alg->cra_exit) alg->cra_exit(tfm); crypto_exit_ops(tfm); - crypto_alg_put(alg); + crypto_mod_put(alg); memset(tfm, 0, size); kfree(tfm); } @@ -243,11 +455,8 @@ static inline int crypto_set_driver_name return 0; } -int crypto_register_alg(struct crypto_alg *alg) +static int crypto_check_alg(struct crypto_alg *alg) { - int ret; - struct crypto_alg *q; - if (alg->cra_alignmask & (alg->cra_alignmask + 1)) return -EINVAL; @@ -260,58 +469,360 @@ int crypto_register_alg(struct crypto_al if (alg->cra_priority < 0) return -EINVAL; - ret = crypto_set_driver_name(alg); - if (unlikely(ret)) - return ret; + return crypto_set_driver_name(alg); +} - down_write(&crypto_alg_sem); - +static void crypto_remove_spawns(struct list_head *spawns, + struct list_head *list) +{ + struct crypto_spawn *spawn, *n; + + list_for_each_entry_safe(spawn, n, spawns, list) { + struct crypto_instance *inst = spawn->inst; + struct crypto_template *tmpl = inst->tmpl; + + list_del_init(&spawn->list); + spawn->alg = NULL; + + if (crypto_is_dead(&inst->alg)) + continue; + + inst->alg.cra_flags |= CRYPTO_ALG_DEAD; + if (!tmpl || !crypto_tmpl_get(tmpl)) + continue; + + crypto_notify(CRYPTO_MSG_ALG_UNREGISTER, &inst->alg); + list_move(&inst->alg.cra_list, list); + hlist_del(&inst->list); + inst->alg.cra_destroy = crypto_destroy_instance; + + if (!list_empty(&inst->alg.cra_users)) { + if (&n->list == spawns) + n = list_entry(inst->alg.cra_users.next, + typeof(*n), list); + __list_splice(&inst->alg.cra_users, spawns->prev); + } + } +} + +static int __crypto_register_alg(struct crypto_alg *alg, + struct list_head *list) +{ + struct crypto_alg *q; + int ret = -EAGAIN; + + if (crypto_is_dead(alg)) + goto out; + + INIT_LIST_HEAD(&alg->cra_users); + + ret = -EEXIST; + atomic_set(&alg->cra_refcnt, 1); list_for_each_entry(q, &crypto_alg_list, cra_list) { - if (q == alg) { - ret = -EEXIST; + if (q == alg) goto out; + + if (crypto_is_moribund(q)) + continue; + + if (crypto_is_larval(q)) { + struct crypto_larval *larval = (void *)q; + + if (strcmp(alg->cra_name, q->cra_name) && + strcmp(alg->cra_driver_name, q->cra_name)) + continue; + + if (larval->adult) + continue; + if ((q->cra_flags ^ alg->cra_flags) & larval->mask) + continue; + if (!crypto_mod_get(alg)) + continue; + + larval->adult = alg; + complete(&larval->completion); + continue; } + + if (strcmp(alg->cra_name, q->cra_name)) + continue; + + if (strcmp(alg->cra_driver_name, q->cra_driver_name) && + q->cra_priority > alg->cra_priority) + continue; + + crypto_remove_spawns(&q->cra_users, list); } list_add(&alg->cra_list, &crypto_alg_list); + + crypto_notify(CRYPTO_MSG_ALG_REGISTER, alg); + ret = 0; + out: - up_write(&crypto_alg_sem); return ret; } +int crypto_register_alg(struct crypto_alg *alg) +{ + LIST_HEAD(list); + int err; + + err = crypto_check_alg(alg); + if (err) + return err; + + down_write(&crypto_alg_sem); + err = __crypto_register_alg(alg, &list); + up_write(&crypto_alg_sem); + + crypto_remove_final(&list); + return err; +} + +static void crypto_destroy_instance(struct crypto_alg *alg) +{ + struct crypto_instance *inst = (void *)alg; + struct crypto_template *tmpl = inst->tmpl; + + tmpl->free(inst); + crypto_tmpl_put(tmpl); +} + +static int crypto_remove_alg(struct crypto_alg *alg, struct list_head *list) +{ + if (unlikely(list_empty(&alg->cra_list))) + return -ENOENT; + + alg->cra_flags |= CRYPTO_ALG_DEAD; + + crypto_notify(CRYPTO_MSG_ALG_UNREGISTER, alg); + list_del_init(&alg->cra_list); + crypto_remove_spawns(&alg->cra_users, list); + + return 0; +} + +static void crypto_remove_final(struct list_head *list) +{ + struct crypto_alg *alg; + struct crypto_alg *n; + + list_for_each_entry_safe(alg, n, list, cra_list) { + list_del_init(&alg->cra_list); + crypto_alg_put(alg); + } +} + int crypto_unregister_alg(struct crypto_alg *alg) { - int ret = -ENOENT; - struct crypto_alg *q; - - BUG_ON(!alg->cra_module); + int ret; + LIST_HEAD(list); down_write(&crypto_alg_sem); - list_for_each_entry(q, &crypto_alg_list, cra_list) { - if (alg == q) { - list_del(&alg->cra_list); - ret = 0; + ret = crypto_remove_alg(alg, &list); + up_write(&crypto_alg_sem); + + if (ret) + return ret; + + BUG_ON(atomic_read(&alg->cra_refcnt) != 1); + if (alg->cra_destroy) + alg->cra_destroy(alg); + + crypto_remove_final(&list); + return 0; +} + +int crypto_register_template(struct crypto_template *tmpl) +{ + struct crypto_template *q; + int err = -EEXIST; + + down_write(&crypto_alg_sem); + + list_for_each_entry(q, &crypto_template_list, list) { + if (q == tmpl) goto out; - } } -out: + + list_add(&tmpl->list, &crypto_template_list); + crypto_notify(CRYPTO_MSG_TMPL_REGISTER, tmpl); + err = 0; +out: up_write(&crypto_alg_sem); - return ret; + return err; } +EXPORT_SYMBOL_GPL(crypto_register_template); + +void crypto_unregister_template(struct crypto_template *tmpl) +{ + struct crypto_instance *inst; + struct hlist_node *p, *n; + struct hlist_head *list; + LIST_HEAD(users); + + down_write(&crypto_alg_sem); + + BUG_ON(list_empty(&tmpl->list)); + list_del_init(&tmpl->list); + + list = &tmpl->instances; + hlist_for_each_entry(inst, p, list, list) { + int err = crypto_remove_alg(&inst->alg, &users); + BUG_ON(err); + } + + crypto_notify(CRYPTO_MSG_TMPL_UNREGISTER, tmpl); + + up_write(&crypto_alg_sem); + + hlist_for_each_entry_safe(inst, p, n, list, list) { + BUG_ON(atomic_read(&inst->alg.cra_refcnt) != 1); + tmpl->free(inst); + } + crypto_remove_final(&users); +} +EXPORT_SYMBOL_GPL(crypto_unregister_template); + +static struct crypto_template *__crypto_lookup_template(const char *name) +{ + struct crypto_template *q, *tmpl = NULL; + + down_read(&crypto_alg_sem); + list_for_each_entry(q, &crypto_template_list, list) { + if (strcmp(q->name, name)) + continue; + if (unlikely(!crypto_tmpl_get(q))) + continue; + + tmpl = q; + break; + } + up_read(&crypto_alg_sem); + + return tmpl; +} + +struct crypto_template *crypto_lookup_template(const char *name) +{ + return try_then_request_module(__crypto_lookup_template(name), name); +} +EXPORT_SYMBOL_GPL(crypto_lookup_template); + +int crypto_register_instance(struct crypto_template *tmpl, + struct crypto_instance *inst) +{ + LIST_HEAD(list); + int err = -EINVAL; + + if (inst->alg.cra_destroy) + goto err; + + err = crypto_check_alg(&inst->alg); + if (err) + goto err; + + inst->alg.cra_module = tmpl->module; + + down_write(&crypto_alg_sem); + + err = __crypto_register_alg(&inst->alg, &list); + if (err) + goto unlock; + + hlist_add_head(&inst->list, &tmpl->instances); + inst->tmpl = tmpl; + +unlock: + up_write(&crypto_alg_sem); + + crypto_remove_final(&list); + +err: + return err; +} +EXPORT_SYMBOL_GPL(crypto_register_instance); + +int crypto_init_spawn(struct crypto_spawn *spawn, struct crypto_alg *alg, + struct crypto_instance *inst) +{ + int err = -EAGAIN; + + spawn->inst = inst; + + down_write(&crypto_alg_sem); + if (!crypto_is_moribund(alg)) { + list_add(&spawn->list, &alg->cra_users); + spawn->alg = alg; + err = 0; + } + up_write(&crypto_alg_sem); + + return err; +} +EXPORT_SYMBOL_GPL(crypto_init_spawn); + +void crypto_drop_spawn(struct crypto_spawn *spawn) +{ + down_write(&crypto_alg_sem); + list_del(&spawn->list); + up_write(&crypto_alg_sem); +} +EXPORT_SYMBOL_GPL(crypto_drop_spawn); + +struct crypto_tfm *crypto_spawn_tfm(struct crypto_spawn *spawn) +{ + struct crypto_alg *alg; + struct crypto_alg *alg2; + struct crypto_tfm *tfm; + + down_read(&crypto_alg_sem); + alg = spawn->alg; + alg2 = alg; + if (alg2) + alg2 = crypto_mod_get(alg2); + up_read(&crypto_alg_sem); + + if (!alg2) { + if (alg) + crypto_shoot_alg(alg); + return ERR_PTR(-EAGAIN); + } + + tfm = __crypto_alloc_tfm(alg, 0); + if (IS_ERR(tfm)) + crypto_mod_put(alg); + + return tfm; +} +EXPORT_SYMBOL_GPL(crypto_spawn_tfm); int crypto_alg_available(const char *name, u32 flags) { int ret = 0; - struct crypto_alg *alg = crypto_alg_mod_lookup(name); + struct crypto_alg *alg = crypto_alg_mod_lookup(name, 0, 0); - if (alg) { - crypto_alg_put(alg); + if (!IS_ERR(alg)) { + crypto_mod_put(alg); ret = 1; } return ret; } +int crypto_register_notifier(struct notifier_block *nb) +{ + return blocking_notifier_chain_register(&crypto_chain, nb); +} +EXPORT_SYMBOL_GPL(crypto_register_notifier); + +int crypto_unregister_notifier(struct notifier_block *nb) +{ + return blocking_notifier_chain_unregister(&crypto_chain, nb); +} +EXPORT_SYMBOL_GPL(crypto_unregister_notifier); + static int __init init_crypto(void) { printk(KERN_INFO "Initializing Cryptographic API\n"); diff --git a/crypto/arc4.c b/crypto/arc4.c index 5edc6a6..8be47e1 100644 --- a/crypto/arc4.c +++ b/crypto/arc4.c @@ -25,7 +25,7 @@ struct arc4_ctx { }; static int arc4_set_key(struct crypto_tfm *tfm, const u8 *in_key, - unsigned int key_len, u32 *flags) + unsigned int key_len) { struct arc4_ctx *ctx = crypto_tfm_ctx(tfm); int i, j = 0, k = 0; diff --git a/crypto/blowfish.c b/crypto/blowfish.c index 490265f..55238c4 100644 --- a/crypto/blowfish.c +++ b/crypto/blowfish.c @@ -399,8 +399,7 @@ static void bf_decrypt(struct crypto_tfm /* * Calculates the blowfish S and P boxes for encryption and decryption. */ -static int bf_setkey(struct crypto_tfm *tfm, const u8 *key, - unsigned int keylen, u32 *flags) +static int bf_setkey(struct crypto_tfm *tfm, const u8 *key, unsigned int keylen) { struct bf_ctx *ctx = crypto_tfm_ctx(tfm); u32 *P = ctx->p; diff --git a/crypto/cast5.c b/crypto/cast5.c index 08eef58..13ea60a 100644 --- a/crypto/cast5.c +++ b/crypto/cast5.c @@ -769,8 +769,7 @@ #undef zi } -static int cast5_setkey(struct crypto_tfm *tfm, const u8 *key, - unsigned key_len, u32 *flags) +static int cast5_setkey(struct crypto_tfm *tfm, const u8 *key, unsigned key_len) { struct cast5_ctx *c = crypto_tfm_ctx(tfm); int i; @@ -778,11 +777,6 @@ static int cast5_setkey(struct crypto_tf u32 z[4]; u32 k[16]; __be32 p_key[4]; - - if (key_len < 5 || key_len > 16) { - *flags |= CRYPTO_TFM_RES_BAD_KEY_LEN; - return -EINVAL; - } c->rr = key_len <= 10 ? 1 : 0; diff --git a/crypto/cast6.c b/crypto/cast6.c index 08e33bf..136ab6d 100644 --- a/crypto/cast6.c +++ b/crypto/cast6.c @@ -382,14 +382,15 @@ static inline void W(u32 *key, unsigned } static int cast6_setkey(struct crypto_tfm *tfm, const u8 *in_key, - unsigned key_len, u32 *flags) + unsigned key_len) { int i; u32 key[8]; __be32 p_key[8]; /* padded key */ struct cast6_ctx *c = crypto_tfm_ctx(tfm); + u32 *flags = &tfm->crt_flags; - if (key_len < 16 || key_len > 32 || key_len % 4 != 0) { + if (key_len % 4 != 0) { *flags |= CRYPTO_TFM_RES_BAD_KEY_LEN; return -EINVAL; } diff --git a/crypto/cipher.c b/crypto/cipher.c index b899eb9..56406a4 100644 --- a/crypto/cipher.c +++ b/crypto/cipher.c @@ -264,12 +264,12 @@ static int setkey(struct crypto_tfm *tfm { struct cipher_alg *cia = &tfm->__crt_alg->cra_cipher; + tfm->crt_flags &= ~CRYPTO_TFM_RES_MASK; if (keylen < cia->cia_min_keysize || keylen > cia->cia_max_keysize) { tfm->crt_flags |= CRYPTO_TFM_RES_BAD_KEY_LEN; return -EINVAL; } else - return cia->cia_setkey(tfm, key, keylen, - &tfm->crt_flags); + return cia->cia_setkey(tfm, key, keylen); } static int ecb_encrypt(struct crypto_tfm *tfm, diff --git a/crypto/crc32c.c b/crypto/crc32c.c index f266012..0fa7443 100644 --- a/crypto/crc32c.c +++ b/crypto/crc32c.c @@ -16,14 +16,14 @@ #include #include #include #include -#include -#include +#include #define CHKSUM_BLOCK_SIZE 32 #define CHKSUM_DIGEST_SIZE 4 struct chksum_ctx { u32 crc; + u32 key; }; /* @@ -35,7 +35,7 @@ static void chksum_init(struct crypto_tf { struct chksum_ctx *mctx = crypto_tfm_ctx(tfm); - mctx->crc = ~(u32)0; /* common usage */ + mctx->crc = mctx->key; } /* @@ -44,16 +44,15 @@ static void chksum_init(struct crypto_tf * the seed. */ static int chksum_setkey(struct crypto_tfm *tfm, const u8 *key, - unsigned int keylen, u32 *flags) + unsigned int keylen) { struct chksum_ctx *mctx = crypto_tfm_ctx(tfm); if (keylen != sizeof(mctx->crc)) { - if (flags) - *flags = CRYPTO_TFM_RES_BAD_KEY_LEN; + tfm->crt_flags |= CRYPTO_TFM_RES_BAD_KEY_LEN; return -EINVAL; } - mctx->crc = __cpu_to_le32(*(u32 *)key); + mctx->key = le32_to_cpu(*(__le32 *)key); return 0; } @@ -61,19 +60,23 @@ static void chksum_update(struct crypto_ unsigned int length) { struct chksum_ctx *mctx = crypto_tfm_ctx(tfm); - u32 mcrc; - mcrc = crc32c(mctx->crc, data, (size_t)length); - - mctx->crc = mcrc; + mctx->crc = crc32c(mctx->crc, data, length); } static void chksum_final(struct crypto_tfm *tfm, u8 *out) { struct chksum_ctx *mctx = crypto_tfm_ctx(tfm); - u32 mcrc = (mctx->crc ^ ~(u32)0); - *(u32 *)out = __le32_to_cpu(mcrc); + *(__le32 *)out = ~cpu_to_le32(mctx->crc); +} + +static int crc32c_cra_init(struct crypto_tfm *tfm) +{ + struct chksum_ctx *mctx = crypto_tfm_ctx(tfm); + + mctx->key = ~0; + return 0; } static struct crypto_alg alg = { @@ -83,6 +86,7 @@ static struct crypto_alg alg = { .cra_ctxsize = sizeof(struct chksum_ctx), .cra_module = THIS_MODULE, .cra_list = LIST_HEAD_INIT(alg.cra_list), + .cra_init = crc32c_cra_init, .cra_u = { .digest = { .dia_digestsize= CHKSUM_DIGEST_SIZE, diff --git a/crypto/crypto_null.c b/crypto/crypto_null.c index a0d956b..24dbb5d 100644 --- a/crypto/crypto_null.c +++ b/crypto/crypto_null.c @@ -48,7 +48,7 @@ static void null_final(struct crypto_tfm { } static int null_setkey(struct crypto_tfm *tfm, const u8 *key, - unsigned int keylen, u32 *flags) + unsigned int keylen) { return 0; } static void null_crypt(struct crypto_tfm *tfm, u8 *dst, const u8 *src) diff --git a/crypto/cryptomgr.c b/crypto/cryptomgr.c new file mode 100644 index 0000000..ebe637c --- /dev/null +++ b/crypto/cryptomgr.c @@ -0,0 +1,120 @@ +/* + * Create default crypto algorithm instances. + * + * Copyright (c) 2006 Herbert Xu + * + * This program is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License as published by the Free + * Software Foundation; either version 2 of the License, or (at your option) + * any later version. + * + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "internal.h" + +struct cryptomgr_param { + struct { + struct rtattr attr; + struct crypto_attr_alg data; + } alg; +}; + +static int cryptomgr_probe(struct crypto_larval *larval) +{ + struct cryptomgr_param param; + struct crypto_template *tmpl; + struct crypto_instance *inst; + const char *name = larval->alg.cra_name; + const char *p; + unsigned int len; + int err; + + for (p = name; isalnum(*p) || *p == '-' || *p == '_'; p++) + ; + + len = p - name; + if (!len || *p != '(') + return NOTIFY_OK; + + memcpy(param.alg.data.name, name, len); + param.alg.data.name[len] = 0; + + name = p + 1; + for (p = name; isalnum(*p) || *p == '-' || *p == '_'; p++) + ; + + len = p - name; + if (!len || *p != ')' || p[1]) + return NOTIFY_OK; + + tmpl = crypto_lookup_template(param.alg.data.name); + if (!tmpl) + goto err; + + param.alg.attr.rta_len = sizeof(param.alg); + param.alg.attr.rta_type = CRYPTOA_ALG; + memcpy(param.alg.data.name, name, len); + param.alg.data.name[len] = 0; + + do { + inst = tmpl->alloc(¶m, sizeof(param)); + if (IS_ERR(inst)) + err = PTR_ERR(inst); + else if ((err = crypto_register_instance(tmpl, inst))) + tmpl->free(inst); + } while (err == -EAGAIN && !signal_pending(current)); + + crypto_tmpl_put(tmpl); + + if (err) + goto err; + + return NOTIFY_STOP; + +err: + crypto_larval_error(larval->alg.cra_name, larval->alg.cra_flags, + larval->mask); + return NOTIFY_STOP; +} + +static int cryptomgr_notify(struct notifier_block *this, unsigned long msg, + void *data) +{ + switch (msg) { + case CRYPTO_MSG_ALG_REQUEST: + return cryptomgr_probe(data); + } + + return NOTIFY_DONE; +} + +static struct notifier_block cryptomgr_notifier = { + .notifier_call = cryptomgr_notify, +}; + +static int __init cryptomgr_init(void) +{ + return crypto_register_notifier(&cryptomgr_notifier); +} + +static void __exit cryptomgr_exit(void) +{ + int err = crypto_unregister_notifier(&cryptomgr_notifier); + BUG_ON(err); +} + +module_init(cryptomgr_init); +module_exit(cryptomgr_exit); + +MODULE_LICENSE("GPL"); +MODULE_DESCRIPTION("Crypto Algorithm Manager"); diff --git a/crypto/des.c b/crypto/des.c index a9d3c23..1df3a71 100644 --- a/crypto/des.c +++ b/crypto/des.c @@ -784,9 +784,10 @@ static void dkey(u32 *pe, const u8 *k) } static int des_setkey(struct crypto_tfm *tfm, const u8 *key, - unsigned int keylen, u32 *flags) + unsigned int keylen) { struct des_ctx *dctx = crypto_tfm_ctx(tfm); + u32 *flags = &tfm->crt_flags; u32 tmp[DES_EXPKEY_WORDS]; int ret; @@ -864,11 +865,12 @@ static void des_decrypt(struct crypto_tf * */ static int des3_ede_setkey(struct crypto_tfm *tfm, const u8 *key, - unsigned int keylen, u32 *flags) + unsigned int keylen) { const u32 *K = (const u32 *)key; struct des3_ede_ctx *dctx = crypto_tfm_ctx(tfm); u32 *expkey = dctx->expkey; + u32 *flags = &tfm->crt_flags; if (unlikely(!((K[0] ^ K[2]) | (K[1] ^ K[3])) || !((K[2] ^ K[4]) | (K[3] ^ K[5])))) diff --git a/crypto/digest.c b/crypto/digest.c index 603006a..765f22a 100644 --- a/crypto/digest.c +++ b/crypto/digest.c @@ -18,11 +18,6 @@ #include #include #include "internal.h" -static void init(struct crypto_tfm *tfm) -{ - tfm->__crt_alg->cra_digest.dia_init(tfm); -} - static void update(struct crypto_tfm *tfm, struct scatterlist *sg, unsigned int nsg) { @@ -66,30 +61,40 @@ static void update(struct crypto_tfm *tf static void final(struct crypto_tfm *tfm, u8 *out) { unsigned long alignmask = crypto_tfm_alg_alignmask(tfm); + struct digest_alg *digest = &tfm->__crt_alg->cra_digest; + if (unlikely((unsigned long)out & alignmask)) { - unsigned int size = crypto_tfm_alg_digestsize(tfm); - u8 buffer[size + alignmask]; - u8 *dst = (u8 *)ALIGN((unsigned long)buffer, alignmask + 1); - tfm->__crt_alg->cra_digest.dia_final(tfm, dst); - memcpy(out, dst, size); + unsigned long align = alignmask + 1; + unsigned long addr = (unsigned long)crypto_tfm_ctx(tfm); + u8 *dst = (u8 *)ALIGN(addr, align) + + ALIGN(tfm->__crt_alg->cra_ctxsize, align); + + digest->dia_final(tfm, dst); + memcpy(out, dst, digest->dia_digestsize); } else - tfm->__crt_alg->cra_digest.dia_final(tfm, out); + digest->dia_final(tfm, out); +} + +static int nosetkey(struct crypto_tfm *tfm, const u8 *key, unsigned int keylen) +{ + tfm->crt_flags &= ~CRYPTO_TFM_RES_MASK; + return -ENOSYS; } static int setkey(struct crypto_tfm *tfm, const u8 *key, unsigned int keylen) { - u32 flags; - if (tfm->__crt_alg->cra_digest.dia_setkey == NULL) - return -ENOSYS; - return tfm->__crt_alg->cra_digest.dia_setkey(tfm, key, keylen, &flags); + tfm->crt_flags &= ~CRYPTO_TFM_RES_MASK; + return tfm->__crt_alg->cra_digest.dia_setkey(tfm, key, keylen); } static void digest(struct crypto_tfm *tfm, struct scatterlist *sg, unsigned int nsg, u8 *out) { - init(tfm); - update(tfm, sg, nsg); - final(tfm, out); + struct digest_tfm *ops = &tfm->crt_digest; + + ops->dit_init(tfm); + ops->dit_update(tfm, sg, nsg); + ops->dit_final(tfm, out); } int crypto_init_digest_flags(struct crypto_tfm *tfm, u32 flags) @@ -100,17 +105,18 @@ int crypto_init_digest_flags(struct cryp int crypto_init_digest_ops(struct crypto_tfm *tfm) { struct digest_tfm *ops = &tfm->crt_digest; + struct digest_alg *dalg = &tfm->__crt_alg->cra_digest; - ops->dit_init = init; - ops->dit_update = update; - ops->dit_final = final; + ops->dit_init = dalg->dia_init; + ops->dit_update = dalg->dia_update_sg ?: update; + ops->dit_final = crypto_tfm_alg_alignmask(tfm) ? + final : dalg->dia_final; ops->dit_digest = digest; - ops->dit_setkey = setkey; + ops->dit_setkey = dalg->dia_setkey ? setkey : nosetkey; - return crypto_alloc_hmac_block(tfm); + return 0; } void crypto_exit_digest_ops(struct crypto_tfm *tfm) { - crypto_free_hmac_block(tfm); } diff --git a/crypto/hmac.c b/crypto/hmac.c index 46120de..6bd3004 100644 --- a/crypto/hmac.c +++ b/crypto/hmac.c @@ -4,121 +4,226 @@ * HMAC: Keyed-Hashing for Message Authentication (RFC2104). * * Copyright (c) 2002 James Morris + * Copyright (c) 2006 Herbert Xu * * The HMAC implementation is derived from USAGI. * Copyright (c) 2002 Kazunori Miyazawa / USAGI * * This program is free software; you can redistribute it and/or modify it * under the terms of the GNU General Public License as published by the Free - * Software Foundation; either version 2 of the License, or (at your option) + * Software Foundation; either version 2 of the License, or (at your option) * any later version. * */ #include +#include +#include #include -#include +#include #include #include #include "internal.h" -static void hash_key(struct crypto_tfm *tfm, u8 *key, unsigned int keylen) -{ - struct scatterlist tmp; - - sg_set_buf(&tmp, key, keylen); - crypto_digest_digest(tfm, &tmp, 1, key); -} +struct crypto_hmac_ctx { + struct crypto_tfm *child; +}; -int crypto_alloc_hmac_block(struct crypto_tfm *tfm) +static inline struct crypto_hmac_ctx *crypto_hmac_ctx(struct crypto_tfm *tfm) { - int ret = 0; - - BUG_ON(!crypto_tfm_alg_blocksize(tfm)); - - tfm->crt_digest.dit_hmac_block = kmalloc(crypto_tfm_alg_blocksize(tfm), - GFP_KERNEL); - if (tfm->crt_digest.dit_hmac_block == NULL) - ret = -ENOMEM; - - return ret; - + return (void *)((char *)crypto_tfm_ctx_aligned(tfm) + + crypto_tfm_alg_blocksize(tfm) * 2); } -void crypto_free_hmac_block(struct crypto_tfm *tfm) +static int crypto_hmac_digest_setkey(struct crypto_tfm *parent, + const u8 *inkey, unsigned int keylen) { - kfree(tfm->crt_digest.dit_hmac_block); + int bs = crypto_tfm_alg_blocksize(parent); + char *key = crypto_tfm_ctx_aligned(parent); + char *pad = key + bs; + struct crypto_hmac_ctx *ctx = (void *)(pad + bs); + struct crypto_tfm *child = ctx->child; + + if (keylen > bs) { + struct scatterlist tmp; + + sg_set_buf(&tmp, (void *)inkey, keylen); + crypto_digest_digest(child, &tmp, 1, pad); + inkey = pad; + keylen = crypto_tfm_alg_digestsize(parent); + } + + memcpy(key, inkey, keylen); + memset(key + keylen, 0, bs - keylen); + return 0; } -void crypto_hmac_init(struct crypto_tfm *tfm, u8 *key, unsigned int *keylen) +static void crypto_hmac_digest_init(struct crypto_tfm *parent) { + int bs = crypto_tfm_alg_blocksize(parent); + char *key = crypto_tfm_ctx_aligned(parent); + char *ipad = key + bs; + struct crypto_hmac_ctx *ctx = (void *)(ipad + bs); + struct crypto_tfm *tfm = ctx->child; unsigned int i; struct scatterlist tmp; - char *ipad = tfm->crt_digest.dit_hmac_block; - - if (*keylen > crypto_tfm_alg_blocksize(tfm)) { - hash_key(tfm, key, *keylen); - *keylen = crypto_tfm_alg_digestsize(tfm); - } - memset(ipad, 0, crypto_tfm_alg_blocksize(tfm)); - memcpy(ipad, key, *keylen); + memcpy(ipad, key, bs); - for (i = 0; i < crypto_tfm_alg_blocksize(tfm); i++) + for (i = 0; i < bs; i++) ipad[i] ^= 0x36; - sg_set_buf(&tmp, ipad, crypto_tfm_alg_blocksize(tfm)); - + sg_set_buf(&tmp, ipad, bs); + crypto_digest_init(tfm); crypto_digest_update(tfm, &tmp, 1); } -void crypto_hmac_update(struct crypto_tfm *tfm, - struct scatterlist *sg, unsigned int nsg) +static void crypto_hmac_digest_update(struct crypto_tfm *parent, + struct scatterlist *sg, unsigned int nsg) { - crypto_digest_update(tfm, sg, nsg); + struct crypto_hmac_ctx *ctx = crypto_hmac_ctx(parent); + crypto_digest_update(ctx->child, sg, nsg); } -void crypto_hmac_final(struct crypto_tfm *tfm, u8 *key, - unsigned int *keylen, u8 *out) +static void crypto_hmac_digest_final(struct crypto_tfm *parent, u8 *out) { + int bs = crypto_tfm_alg_blocksize(parent); + char *key = crypto_tfm_ctx(parent); + char *opad = key + bs; + struct crypto_hmac_ctx *ctx = (void *)(opad + bs); + struct crypto_tfm *tfm = ctx->child; unsigned int i; struct scatterlist tmp; - char *opad = tfm->crt_digest.dit_hmac_block; - - if (*keylen > crypto_tfm_alg_blocksize(tfm)) { - hash_key(tfm, key, *keylen); - *keylen = crypto_tfm_alg_digestsize(tfm); - } crypto_digest_final(tfm, out); - memset(opad, 0, crypto_tfm_alg_blocksize(tfm)); - memcpy(opad, key, *keylen); - - for (i = 0; i < crypto_tfm_alg_blocksize(tfm); i++) + memcpy(opad, key, bs); + + for (i = 0; i < bs; i++) opad[i] ^= 0x5c; - sg_set_buf(&tmp, opad, crypto_tfm_alg_blocksize(tfm)); + sg_set_buf(&tmp, opad, bs); crypto_digest_init(tfm); crypto_digest_update(tfm, &tmp, 1); - + sg_set_buf(&tmp, out, crypto_tfm_alg_digestsize(tfm)); - + crypto_digest_update(tfm, &tmp, 1); crypto_digest_final(tfm, out); } -void crypto_hmac(struct crypto_tfm *tfm, u8 *key, unsigned int *keylen, - struct scatterlist *sg, unsigned int nsg, u8 *out) +static int crypto_hmac_init_tfm(struct crypto_tfm *tfm) +{ + struct crypto_instance *inst = (void *)tfm->__crt_alg; + struct crypto_spawn *spawn = crypto_instance_ctx(inst); + struct crypto_hmac_ctx *ctx = crypto_hmac_ctx(tfm); + + ctx->child = crypto_spawn_tfm(spawn); + if (IS_ERR(ctx->child)) + return PTR_ERR(ctx->child); + + return 0; +} + +static void crypto_hmac_exit_tfm(struct crypto_tfm *tfm) +{ + crypto_free_tfm(crypto_hmac_ctx(tfm)->child); +} + +static struct crypto_instance *crypto_hmac_alloc(void *param, unsigned int len) +{ + struct rtattr *rta = param; + struct crypto_instance *inst; + struct crypto_attr_alg *alga; + struct crypto_alg *alg; + struct crypto_spawn *spawn; + int err; + + if (!RTA_OK(rta, len)) + return ERR_PTR(-EBADR); + if (rta->rta_type != CRYPTOA_ALG || RTA_PAYLOAD(rta) < sizeof(*alga)) + return ERR_PTR(-EINVAL); + + alga = RTA_DATA(rta); + + inst = kzalloc(sizeof(*inst) + sizeof(*spawn), GFP_KERNEL); + if (!inst) + return ERR_PTR(-ENOMEM); + + inst->alg.cra_flags = CRYPTO_ALG_TYPE_DIGEST; + + alg = crypto_alg_mod_lookup(alga->name, CRYPTO_ALG_TYPE_DIGEST, + CRYPTO_ALG_TYPE_MASK); + err = PTR_ERR(alg); + if (IS_ERR(alg)) + goto err_free_inst; + + err = -ENAMETOOLONG; + if (snprintf(inst->alg.cra_name, CRYPTO_MAX_ALG_NAME, + "hmac(%s)", alg->cra_name) >= CRYPTO_MAX_ALG_NAME) + goto put_alg; + + if (snprintf(inst->alg.cra_driver_name, CRYPTO_MAX_ALG_NAME, + "hmac(%s)", alg->cra_driver_name) >= CRYPTO_MAX_ALG_NAME) + goto put_alg; + + spawn = crypto_instance_ctx(inst); + err = crypto_init_spawn(spawn, alg, inst); + +put_alg: + crypto_mod_put(alg); + if (err) + goto err_free_inst; + + inst->alg.cra_priority = alg->cra_priority; + inst->alg.cra_blocksize = alg->cra_blocksize; + inst->alg.cra_alignmask = alg->cra_alignmask; + inst->alg.cra_digest.dia_digestsize = alg->cra_digest.dia_digestsize; + + inst->alg.cra_ctxsize = sizeof(struct crypto_hmac_ctx) + + inst->alg.cra_blocksize * 2; + + inst->alg.cra_init = crypto_hmac_init_tfm; + inst->alg.cra_exit = crypto_hmac_exit_tfm; + + inst->alg.cra_digest.dia_init = crypto_hmac_digest_init; + inst->alg.cra_digest.dia_update_sg = crypto_hmac_digest_update; + inst->alg.cra_digest.dia_final = crypto_hmac_digest_final; + inst->alg.cra_digest.dia_setkey = crypto_hmac_digest_setkey; + + return inst; + +err_free_inst: + kfree(inst); + return ERR_PTR(err); +} + +static void crypto_hmac_free(struct crypto_instance *inst) +{ + crypto_drop_spawn(crypto_instance_ctx(inst)); + kfree(inst); +} + +static struct crypto_template crypto_hmac_tmpl = { + .name = "hmac", + .alloc = crypto_hmac_alloc, + .free = crypto_hmac_free, + .module = THIS_MODULE, +}; + +static int __init crypto_hmac_module_init(void) +{ + return crypto_register_template(&crypto_hmac_tmpl); +} + +static void __exit crypto_hmac_module_exit(void) { - crypto_hmac_init(tfm, key, keylen); - crypto_hmac_update(tfm, sg, nsg); - crypto_hmac_final(tfm, key, keylen, out); + crypto_unregister_template(&crypto_hmac_tmpl); } -EXPORT_SYMBOL_GPL(crypto_hmac_init); -EXPORT_SYMBOL_GPL(crypto_hmac_update); -EXPORT_SYMBOL_GPL(crypto_hmac_final); -EXPORT_SYMBOL_GPL(crypto_hmac); +module_init(crypto_hmac_module_init); +module_exit(crypto_hmac_module_exit); +MODULE_LICENSE("GPL"); +MODULE_DESCRIPTION("HMAC digest algorithm"); diff --git a/crypto/internal.h b/crypto/internal.h index 959e602..84019f7 100644 --- a/crypto/internal.h +++ b/crypto/internal.h @@ -12,17 +12,65 @@ */ #ifndef _CRYPTO_INTERNAL_H #define _CRYPTO_INTERNAL_H + +#include #include #include #include #include #include #include +#include #include #include #include +#include #include +/* Crypto notification events. */ +enum { + CRYPTO_MSG_ALG_REQUEST, + CRYPTO_MSG_ALG_REGISTER, + CRYPTO_MSG_ALG_UNREGISTER, + CRYPTO_MSG_TMPL_REGISTER, + CRYPTO_MSG_TMPL_UNREGISTER, +}; + +struct notifier_block; + +struct crypto_instance { + struct crypto_alg alg; + + struct crypto_template *tmpl; + struct hlist_node list; + + char __ctx[] __attribute__ ((__aligned__)); +}; + +struct crypto_template { + struct list_head list; + struct hlist_head instances; + struct module *module; + + struct crypto_instance *(*alloc)(void *param, unsigned int len); + void (*free)(struct crypto_instance *inst); + + char name[CRYPTO_MAX_ALG_NAME]; +}; + +struct crypto_larval { + struct crypto_alg alg; + struct crypto_alg *adult; + struct completion completion; + u32 mask; +}; + +struct crypto_spawn { + struct list_head list; + struct crypto_alg *alg; + struct crypto_instance *inst; +}; + extern struct list_head crypto_alg_list; extern struct rw_semaphore crypto_alg_sem; @@ -49,19 +97,6 @@ static inline void crypto_yield(struct c cond_resched(); } -#ifdef CONFIG_CRYPTO_HMAC -int crypto_alloc_hmac_block(struct crypto_tfm *tfm); -void crypto_free_hmac_block(struct crypto_tfm *tfm); -#else -static inline int crypto_alloc_hmac_block(struct crypto_tfm *tfm) -{ - return 0; -} - -static inline void crypto_free_hmac_block(struct crypto_tfm *tfm) -{ } -#endif - #ifdef CONFIG_PROC_FS void __init crypto_init_proc(void); #else @@ -72,7 +107,14 @@ #endif static inline unsigned int crypto_digest_ctxsize(struct crypto_alg *alg, int flags) { - return alg->cra_ctxsize; + unsigned int len = alg->cra_ctxsize; + + if (alg->cra_alignmask) { + len = ALIGN(len, (unsigned long)alg->cra_alignmask + 1); + len += alg->cra_digest.dia_digestsize; + } + + return len; } static inline unsigned int crypto_cipher_ctxsize(struct crypto_alg *alg, @@ -108,5 +150,38 @@ void crypto_exit_digest_ops(struct crypt void crypto_exit_cipher_ops(struct crypto_tfm *tfm); void crypto_exit_compress_ops(struct crypto_tfm *tfm); +void crypto_mod_put(struct crypto_alg *alg); +void crypto_larval_error(const char *name, u32 type, u32 mask); +struct crypto_alg *crypto_alg_mod_lookup(const char *name, u32 type, u32 mask); + +int crypto_register_template(struct crypto_template *tmpl); +void crypto_unregister_template(struct crypto_template *tmpl); +struct crypto_template *crypto_lookup_template(const char *name); +int crypto_register_instance(struct crypto_template *tmpl, + struct crypto_instance *inst); + +int crypto_register_notifier(struct notifier_block *nb); +int crypto_unregister_notifier(struct notifier_block *nb); + +int crypto_init_spawn(struct crypto_spawn *spawn, struct crypto_alg *alg, + struct crypto_instance *inst); +void crypto_drop_spawn(struct crypto_spawn *spawn); +struct crypto_tfm *crypto_spawn_tfm(struct crypto_spawn *spawn); + +static inline void *crypto_instance_ctx(struct crypto_instance *inst) +{ + return inst->__ctx; +} + +static inline int crypto_tmpl_get(struct crypto_template *tmpl) +{ + return try_module_get(tmpl->module); +} + +static inline void crypto_tmpl_put(struct crypto_template *tmpl) +{ + module_put(tmpl->module); +} + #endif /* _CRYPTO_INTERNAL_H */ diff --git a/crypto/khazad.c b/crypto/khazad.c index d4c9d36..9fa24a2 100644 --- a/crypto/khazad.c +++ b/crypto/khazad.c @@ -755,19 +755,13 @@ static const u64 c[KHAZAD_ROUNDS + 1] = }; static int khazad_setkey(struct crypto_tfm *tfm, const u8 *in_key, - unsigned int key_len, u32 *flags) + unsigned int key_len) { struct khazad_ctx *ctx = crypto_tfm_ctx(tfm); const __be32 *key = (const __be32 *)in_key; int r; const u64 *S = T7; u64 K2, K1; - - if (key_len != 16) - { - *flags |= CRYPTO_TFM_RES_BAD_KEY_LEN; - return -EINVAL; - } /* key is supposed to be 32-bit aligned */ K2 = ((u64)be32_to_cpu(key[0]) << 32) | be32_to_cpu(key[1]); diff --git a/crypto/michael_mic.c b/crypto/michael_mic.c index d061da2..094397b 100644 --- a/crypto/michael_mic.c +++ b/crypto/michael_mic.c @@ -123,14 +123,13 @@ static void michael_final(struct crypto_ static int michael_setkey(struct crypto_tfm *tfm, const u8 *key, - unsigned int keylen, u32 *flags) + unsigned int keylen) { struct michael_mic_ctx *mctx = crypto_tfm_ctx(tfm); const __le32 *data = (const __le32 *)key; if (keylen != 8) { - if (flags) - *flags = CRYPTO_TFM_RES_BAD_KEY_LEN; + tfm->crt_flags |= CRYPTO_TFM_RES_BAD_KEY_LEN; return -EINVAL; } diff --git a/crypto/proc.c b/crypto/proc.c index c0a5dd7..8543b7a 100644 --- a/crypto/proc.c +++ b/crypto/proc.c @@ -12,6 +12,8 @@ * any later version. * */ + +#include #include #include #include @@ -54,6 +56,7 @@ static int c_show(struct seq_file *m, vo seq_printf(m, "driver : %s\n", alg->cra_driver_name); seq_printf(m, "module : %s\n", module_name(alg->cra_module)); seq_printf(m, "priority : %d\n", alg->cra_priority); + seq_printf(m, "refcnt : %d\n", atomic_read(&alg->cra_refcnt)); switch (alg->cra_flags & CRYPTO_ALG_TYPE_MASK) { case CRYPTO_ALG_TYPE_CIPHER: diff --git a/crypto/serpent.c b/crypto/serpent.c index de60cdd..465d091 100644 --- a/crypto/serpent.c +++ b/crypto/serpent.c @@ -216,7 +216,7 @@ struct serpent_ctx { static int serpent_setkey(struct crypto_tfm *tfm, const u8 *key, - unsigned int keylen, u32 *flags) + unsigned int keylen) { struct serpent_ctx *ctx = crypto_tfm_ctx(tfm); u32 *k = ctx->expkey; @@ -224,13 +224,6 @@ static int serpent_setkey(struct crypto_ u32 r0,r1,r2,r3,r4; int i; - if ((keylen < SERPENT_MIN_KEY_SIZE) - || (keylen > SERPENT_MAX_KEY_SIZE)) - { - *flags |= CRYPTO_TFM_RES_BAD_KEY_LEN; - return -EINVAL; - } - /* Copy key, add padding */ for (i = 0; i < keylen; ++i) @@ -497,21 +490,15 @@ static struct crypto_alg serpent_alg = { }; static int tnepres_setkey(struct crypto_tfm *tfm, const u8 *key, - unsigned int keylen, u32 *flags) + unsigned int keylen) { u8 rev_key[SERPENT_MAX_KEY_SIZE]; int i; - if ((keylen < SERPENT_MIN_KEY_SIZE) - || (keylen > SERPENT_MAX_KEY_SIZE)) { - *flags |= CRYPTO_TFM_RES_BAD_KEY_LEN; - return -EINVAL; - } - for (i = 0; i < keylen; ++i) rev_key[keylen - i - 1] = key[i]; - return serpent_setkey(tfm, rev_key, keylen, flags); + return serpent_setkey(tfm, rev_key, keylen); } static void tnepres_encrypt(struct crypto_tfm *tfm, u8 *dst, const u8 *src) diff --git a/crypto/sha1.c b/crypto/sha1.c index 6c77b68..1bba551 100644 --- a/crypto/sha1.c +++ b/crypto/sha1.c @@ -109,6 +109,7 @@ static void sha1_final(struct crypto_tfm static struct crypto_alg alg = { .cra_name = "sha1", + .cra_driver_name= "sha1-generic", .cra_flags = CRYPTO_ALG_TYPE_DIGEST, .cra_blocksize = SHA1_HMAC_BLOCK_SIZE, .cra_ctxsize = sizeof(struct sha1_ctx), @@ -137,3 +138,5 @@ module_exit(fini); MODULE_LICENSE("GPL"); MODULE_DESCRIPTION("SHA1 Secure Hash Algorithm"); + +MODULE_ALIAS("sha1-generic"); diff --git a/crypto/sha256.c b/crypto/sha256.c index bc71d85..716195b 100644 --- a/crypto/sha256.c +++ b/crypto/sha256.c @@ -309,6 +309,7 @@ static void sha256_final(struct crypto_t static struct crypto_alg alg = { .cra_name = "sha256", + .cra_driver_name= "sha256-generic", .cra_flags = CRYPTO_ALG_TYPE_DIGEST, .cra_blocksize = SHA256_HMAC_BLOCK_SIZE, .cra_ctxsize = sizeof(struct sha256_ctx), @@ -337,3 +338,5 @@ module_exit(fini); MODULE_LICENSE("GPL"); MODULE_DESCRIPTION("SHA256 Secure Hash Algorithm"); + +MODULE_ALIAS("sha256-generic"); diff --git a/crypto/tcrypt.c b/crypto/tcrypt.c index e52f56c..f778e40 100644 --- a/crypto/tcrypt.c +++ b/crypto/tcrypt.c @@ -117,13 +117,10 @@ static void test_hash(char *algo, struct sg_set_buf(&sg[0], hash_tv[i].plaintext, hash_tv[i].psize); - crypto_digest_init(tfm); - if (tfm->crt_u.digest.dit_setkey) { + if (hash_tv[i].ksize) crypto_digest_setkey(tfm, hash_tv[i].key, hash_tv[i].ksize); - } - crypto_digest_update(tfm, sg, 1); - crypto_digest_final(tfm, result); + crypto_digest_digest(tfm, sg, 1, result); hexdump(result, crypto_tfm_alg_digestsize(tfm)); printk("%s\n", @@ -154,6 +151,9 @@ static void test_hash(char *algo, struct hash_tv[i].tap[k]); } + if (hash_tv[i].ksize) + crypto_digest_setkey(tfm, hash_tv[i].key, + hash_tv[i].ksize); crypto_digest_digest(tfm, sg, hash_tv[i].np, result); hexdump(result, crypto_tfm_alg_digestsize(tfm)); @@ -167,92 +167,6 @@ static void test_hash(char *algo, struct crypto_free_tfm(tfm); } - -#ifdef CONFIG_CRYPTO_HMAC - -static void test_hmac(char *algo, struct hmac_testvec *template, - unsigned int tcount) -{ - unsigned int i, j, k, temp; - struct scatterlist sg[8]; - char result[64]; - struct crypto_tfm *tfm; - struct hmac_testvec *hmac_tv; - unsigned int tsize, klen; - - tfm = crypto_alloc_tfm(algo, 0); - if (tfm == NULL) { - printk("failed to load transform for %s\n", algo); - return; - } - - printk("\ntesting hmac_%s\n", algo); - - tsize = sizeof(struct hmac_testvec); - tsize *= tcount; - if (tsize > TVMEMSIZE) { - printk("template (%u) too big for tvmem (%u)\n", tsize, - TVMEMSIZE); - goto out; - } - - memcpy(tvmem, template, tsize); - hmac_tv = (void *)tvmem; - - for (i = 0; i < tcount; i++) { - printk("test %u:\n", i + 1); - memset(result, 0, sizeof (result)); - - klen = hmac_tv[i].ksize; - sg_set_buf(&sg[0], hmac_tv[i].plaintext, hmac_tv[i].psize); - - crypto_hmac(tfm, hmac_tv[i].key, &klen, sg, 1, result); - - hexdump(result, crypto_tfm_alg_digestsize(tfm)); - printk("%s\n", - memcmp(result, hmac_tv[i].digest, - crypto_tfm_alg_digestsize(tfm)) ? "fail" : - "pass"); - } - - printk("\ntesting hmac_%s across pages\n", algo); - - memset(xbuf, 0, XBUFSIZE); - - j = 0; - for (i = 0; i < tcount; i++) { - if (hmac_tv[i].np) { - j++; - printk("test %u:\n",j); - memset(result, 0, 64); - - temp = 0; - klen = hmac_tv[i].ksize; - for (k = 0; k < hmac_tv[i].np; k++) { - memcpy(&xbuf[IDX[k]], - hmac_tv[i].plaintext + temp, - hmac_tv[i].tap[k]); - temp += hmac_tv[i].tap[k]; - sg_set_buf(&sg[k], &xbuf[IDX[k]], - hmac_tv[i].tap[k]); - } - - crypto_hmac(tfm, hmac_tv[i].key, &klen, sg, - hmac_tv[i].np, result); - hexdump(result, crypto_tfm_alg_digestsize(tfm)); - - printk("%s\n", - memcmp(result, hmac_tv[i].digest, - crypto_tfm_alg_digestsize(tfm)) ? - "fail" : "pass"); - } - } -out: - crypto_free_tfm(tfm); -} - -#endif /* CONFIG_CRYPTO_HMAC */ - static void test_cipher(char *algo, int mode, int enc, struct cipher_testvec *template, unsigned int tcount) { @@ -765,104 +679,6 @@ out: crypto_free_tfm(tfm); } -static void test_crc32c(void) -{ -#define NUMVEC 6 -#define VECSIZE 40 - - int i, j, pass; - u32 crc; - u8 b, test_vec[NUMVEC][VECSIZE]; - static u32 vec_results[NUMVEC] = { - 0x0e2c157f, 0xe980ebf6, 0xde74bded, - 0xd579c862, 0xba979ad0, 0x2b29d913 - }; - static u32 tot_vec_results = 0x24c5d375; - - struct scatterlist sg[NUMVEC]; - struct crypto_tfm *tfm; - char *fmtdata = "testing crc32c initialized to %08x: %s\n"; -#define SEEDTESTVAL 0xedcba987 - u32 seed; - - printk("\ntesting crc32c\n"); - - tfm = crypto_alloc_tfm("crc32c", 0); - if (tfm == NULL) { - printk("failed to load transform for crc32c\n"); - return; - } - - crypto_digest_init(tfm); - crypto_digest_final(tfm, (u8*)&crc); - printk(fmtdata, crc, (crc == 0) ? "pass" : "ERROR"); - - /* - * stuff test_vec with known values, simple incrementing - * byte values. - */ - b = 0; - for (i = 0; i < NUMVEC; i++) { - for (j = 0; j < VECSIZE; j++) - test_vec[i][j] = ++b; - sg_set_buf(&sg[i], test_vec[i], VECSIZE); - } - - seed = SEEDTESTVAL; - (void)crypto_digest_setkey(tfm, (const u8*)&seed, sizeof(u32)); - crypto_digest_final(tfm, (u8*)&crc); - printk("testing crc32c setkey returns %08x : %s\n", crc, (crc == (SEEDTESTVAL ^ ~(u32)0)) ? - "pass" : "ERROR"); - - printk("testing crc32c using update/final:\n"); - - pass = 1; /* assume all is well */ - - for (i = 0; i < NUMVEC; i++) { - seed = ~(u32)0; - (void)crypto_digest_setkey(tfm, (const u8*)&seed, sizeof(u32)); - crypto_digest_update(tfm, &sg[i], 1); - crypto_digest_final(tfm, (u8*)&crc); - if (crc == vec_results[i]) { - printk(" %08x:OK", crc); - } else { - printk(" %08x:BAD, wanted %08x\n", crc, vec_results[i]); - pass = 0; - } - } - - printk("\ntesting crc32c using incremental accumulator:\n"); - crc = 0; - for (i = 0; i < NUMVEC; i++) { - seed = (crc ^ ~(u32)0); - (void)crypto_digest_setkey(tfm, (const u8*)&seed, sizeof(u32)); - crypto_digest_update(tfm, &sg[i], 1); - crypto_digest_final(tfm, (u8*)&crc); - } - if (crc == tot_vec_results) { - printk(" %08x:OK", crc); - } else { - printk(" %08x:BAD, wanted %08x\n", crc, tot_vec_results); - pass = 0; - } - - printk("\ntesting crc32c using digest:\n"); - seed = ~(u32)0; - (void)crypto_digest_setkey(tfm, (const u8*)&seed, sizeof(u32)); - crypto_digest_digest(tfm, sg, NUMVEC, (u8*)&crc); - if (crc == tot_vec_results) { - printk(" %08x:OK", crc); - } else { - printk(" %08x:BAD, wanted %08x\n", crc, tot_vec_results); - pass = 0; - } - - printk("\n%s\n", pass ? "pass" : "ERROR"); - - crypto_free_tfm(tfm); - printk("crc32c test complete\n"); -} - static void test_available(void) { char **name = check; @@ -968,12 +784,13 @@ static void do_test(void) test_hash("tgr160", tgr160_tv_template, TGR160_TEST_VECTORS); test_hash("tgr128", tgr128_tv_template, TGR128_TEST_VECTORS); test_deflate(); - test_crc32c(); -#ifdef CONFIG_CRYPTO_HMAC - test_hmac("md5", hmac_md5_tv_template, HMAC_MD5_TEST_VECTORS); - test_hmac("sha1", hmac_sha1_tv_template, HMAC_SHA1_TEST_VECTORS); - test_hmac("sha256", hmac_sha256_tv_template, HMAC_SHA256_TEST_VECTORS); -#endif + test_hash("crc32c", crc32c_tv_template, CRC32C_TEST_VECTORS); + test_hash("hmac(md5)", hmac_md5_tv_template, + HMAC_MD5_TEST_VECTORS); + test_hash("hmac(sha1)", hmac_sha1_tv_template, + HMAC_SHA1_TEST_VECTORS); + test_hash("hmac(sha256)", hmac_sha256_tv_template, + HMAC_SHA256_TEST_VECTORS); test_hash("michael_mic", michael_mic_tv_template, MICHAEL_MIC_TEST_VECTORS); break; @@ -1064,7 +881,7 @@ #endif break; case 18: - test_crc32c(); + test_hash("crc32c", crc32c_tv_template, CRC32C_TEST_VECTORS); break; case 19: @@ -1124,20 +941,21 @@ #endif test_cipher ("xeta", MODE_ECB, DECRYPT, xeta_dec_tv_template, XETA_DEC_TEST_VECTORS); break; -#ifdef CONFIG_CRYPTO_HMAC case 100: - test_hmac("md5", hmac_md5_tv_template, HMAC_MD5_TEST_VECTORS); + test_hash("hmac(md5)", hmac_md5_tv_template, + HMAC_MD5_TEST_VECTORS); break; case 101: - test_hmac("sha1", hmac_sha1_tv_template, HMAC_SHA1_TEST_VECTORS); + test_hash("hmac(sha1)", hmac_sha1_tv_template, + HMAC_SHA1_TEST_VECTORS); break; case 102: - test_hmac("sha256", hmac_sha256_tv_template, HMAC_SHA256_TEST_VECTORS); + test_hash("hmac(sha256)", hmac_sha256_tv_template, + HMAC_SHA256_TEST_VECTORS); break; -#endif case 200: test_cipher_speed("aes", MODE_ECB, ENCRYPT, sec, NULL, 0, diff --git a/crypto/tcrypt.h b/crypto/tcrypt.h index 1fac560..dd648f2 100644 --- a/crypto/tcrypt.h +++ b/crypto/tcrypt.h @@ -28,7 +28,7 @@ #define MAX_IVLEN 32 struct hash_testvec { /* only used with keyed hash algorithms */ char key[128] __attribute__ ((__aligned__(4))); - char plaintext[128]; + char plaintext[240]; char digest[MAX_DIGEST_SIZE]; unsigned char tap[MAX_TAP]; unsigned char psize; @@ -36,16 +36,6 @@ struct hash_testvec { unsigned char ksize; }; -struct hmac_testvec { - char key[128]; - char plaintext[128]; - char digest[MAX_DIGEST_SIZE]; - unsigned char tap[MAX_TAP]; - unsigned char ksize; - unsigned char psize; - unsigned char np; -}; - struct cipher_testvec { char key[MAX_KEYLEN] __attribute__ ((__aligned__(4))); char iv[MAX_IVLEN]; @@ -697,14 +687,13 @@ static struct hash_testvec tgr128_tv_tem }, }; -#ifdef CONFIG_CRYPTO_HMAC /* * HMAC-MD5 test vectors from RFC2202 * (These need to be fixed to not use strlen). */ #define HMAC_MD5_TEST_VECTORS 7 -static struct hmac_testvec hmac_md5_tv_template[] = +static struct hash_testvec hmac_md5_tv_template[] = { { .key = { [0 ... 15] = 0x0b }, @@ -768,7 +757,7 @@ static struct hmac_testvec hmac_md5_tv_t */ #define HMAC_SHA1_TEST_VECTORS 7 -static struct hmac_testvec hmac_sha1_tv_template[] = { +static struct hash_testvec hmac_sha1_tv_template[] = { { .key = { [0 ... 19] = 0x0b }, .ksize = 20, @@ -833,7 +822,7 @@ static struct hmac_testvec hmac_sha1_tv_ */ #define HMAC_SHA256_TEST_VECTORS 10 -static struct hmac_testvec hmac_sha256_tv_template[] = { +static struct hash_testvec hmac_sha256_tv_template[] = { { .key = { 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f, 0x10, @@ -944,8 +933,6 @@ static struct hmac_testvec hmac_sha256_t }, }; -#endif /* CONFIG_CRYPTO_HMAC */ - /* * DES test vectors. */ @@ -2897,6 +2884,183 @@ static struct hash_testvec michael_mic_t }; /* + * CRC32C test vectors + */ +#define CRC32C_TEST_VECTORS 14 + +static struct hash_testvec crc32c_tv_template[] = { + { + .psize = 0, + .digest = { 0x00, 0x00, 0x00, 0x00 } + }, + { + .key = { 0x87, 0xa9, 0xcb, 0xed }, + .ksize = 4, + .psize = 0, + .digest = { 0x78, 0x56, 0x34, 0x12 }, + }, + { + .key = { 0xff, 0xff, 0xff, 0xff }, + .ksize = 4, + .plaintext = { 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, + 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f, 0x10, + 0x11, 0x12, 0x13, 0x14, 0x15, 0x16, 0x17, 0x18, + 0x19, 0x1a, 0x1b, 0x1c, 0x1d, 0x1e, 0x1f, 0x20, + 0x21, 0x22, 0x23, 0x24, 0x25, 0x26, 0x27, 0x28 }, + .psize = 40, + .digest = { 0x7f, 0x15, 0x2c, 0x0e } + }, + { + .key = { 0xff, 0xff, 0xff, 0xff }, + .ksize = 4, + .plaintext = { 0x29, 0x2a, 0x2b, 0x2c, 0x2d, 0x2e, 0x2f, 0x30, + 0x31, 0x32, 0x33, 0x34, 0x35, 0x36, 0x37, 0x38, + 0x39, 0x3a, 0x3b, 0x3c, 0x3d, 0x3e, 0x3f, 0x40, + 0x41, 0x42, 0x43, 0x44, 0x45, 0x46, 0x47, 0x48, + 0x49, 0x4a, 0x4b, 0x4c, 0x4d, 0x4e, 0x4f, 0x50 }, + .psize = 40, + .digest = { 0xf6, 0xeb, 0x80, 0xe9 } + }, + { + .key = { 0xff, 0xff, 0xff, 0xff }, + .ksize = 4, + .plaintext = { 0x51, 0x52, 0x53, 0x54, 0x55, 0x56, 0x57, 0x58, + 0x59, 0x5a, 0x5b, 0x5c, 0x5d, 0x5e, 0x5f, 0x60, + 0x61, 0x62, 0x63, 0x64, 0x65, 0x66, 0x67, 0x68, + 0x69, 0x6a, 0x6b, 0x6c, 0x6d, 0x6e, 0x6f, 0x70, + 0x71, 0x72, 0x73, 0x74, 0x75, 0x76, 0x77, 0x78 }, + .psize = 40, + .digest = { 0xed, 0xbd, 0x74, 0xde } + }, + { + .key = { 0xff, 0xff, 0xff, 0xff }, + .ksize = 4, + .plaintext = { 0x79, 0x7a, 0x7b, 0x7c, 0x7d, 0x7e, 0x7f, 0x80, + 0x81, 0x82, 0x83, 0x84, 0x85, 0x86, 0x87, 0x88, + 0x89, 0x8a, 0x8b, 0x8c, 0x8d, 0x8e, 0x8f, 0x90, + 0x91, 0x92, 0x93, 0x94, 0x95, 0x96, 0x97, 0x98, + 0x99, 0x9a, 0x9b, 0x9c, 0x9d, 0x9e, 0x9f, 0xa0 }, + .psize = 40, + .digest = { 0x62, 0xc8, 0x79, 0xd5 } + }, + { + .key = { 0xff, 0xff, 0xff, 0xff }, + .ksize = 4, + .plaintext = { 0xa1, 0xa2, 0xa3, 0xa4, 0xa5, 0xa6, 0xa7, 0xa8, + 0xa9, 0xaa, 0xab, 0xac, 0xad, 0xae, 0xaf, 0xb0, + 0xb1, 0xb2, 0xb3, 0xb4, 0xb5, 0xb6, 0xb7, 0xb8, + 0xb9, 0xba, 0xbb, 0xbc, 0xbd, 0xbe, 0xbf, 0xc0, + 0xc1, 0xc2, 0xc3, 0xc4, 0xc5, 0xc6, 0xc7, 0xc8 }, + .psize = 40, + .digest = { 0xd0, 0x9a, 0x97, 0xba } + }, + { + .key = { 0xff, 0xff, 0xff, 0xff }, + .ksize = 4, + .plaintext = { 0xc9, 0xca, 0xcb, 0xcc, 0xcd, 0xce, 0xcf, 0xd0, + 0xd1, 0xd2, 0xd3, 0xd4, 0xd5, 0xd6, 0xd7, 0xd8, + 0xd9, 0xda, 0xdb, 0xdc, 0xdd, 0xde, 0xdf, 0xe0, + 0xe1, 0xe2, 0xe3, 0xe4, 0xe5, 0xe6, 0xe7, 0xe8, + 0xe9, 0xea, 0xeb, 0xec, 0xed, 0xee, 0xef, 0xf0 }, + .psize = 40, + .digest = { 0x13, 0xd9, 0x29, 0x2b } + }, + { + .key = { 0x7f, 0x15, 0x2c, 0x0e }, + .ksize = 4, + .plaintext = { 0x29, 0x2a, 0x2b, 0x2c, 0x2d, 0x2e, 0x2f, 0x30, + 0x31, 0x32, 0x33, 0x34, 0x35, 0x36, 0x37, 0x38, + 0x39, 0x3a, 0x3b, 0x3c, 0x3d, 0x3e, 0x3f, 0x40, + 0x41, 0x42, 0x43, 0x44, 0x45, 0x46, 0x47, 0x48, + 0x49, 0x4a, 0x4b, 0x4c, 0x4d, 0x4e, 0x4f, 0x50 }, + .psize = 40, + .digest = { 0x0c, 0xb5, 0xe2, 0xa2 } + }, + { + .key = { 0xf3, 0x4a, 0x1d, 0x5d }, + .ksize = 4, + .plaintext = { 0x51, 0x52, 0x53, 0x54, 0x55, 0x56, 0x57, 0x58, + 0x59, 0x5a, 0x5b, 0x5c, 0x5d, 0x5e, 0x5f, 0x60, + 0x61, 0x62, 0x63, 0x64, 0x65, 0x66, 0x67, 0x68, + 0x69, 0x6a, 0x6b, 0x6c, 0x6d, 0x6e, 0x6f, 0x70, + 0x71, 0x72, 0x73, 0x74, 0x75, 0x76, 0x77, 0x78 }, + .psize = 40, + .digest = { 0xd1, 0x7f, 0xfb, 0xa6 } + }, + { + .key = { 0x2e, 0x80, 0x04, 0x59 }, + .ksize = 4, + .plaintext = { 0x79, 0x7a, 0x7b, 0x7c, 0x7d, 0x7e, 0x7f, 0x80, + 0x81, 0x82, 0x83, 0x84, 0x85, 0x86, 0x87, 0x88, + 0x89, 0x8a, 0x8b, 0x8c, 0x8d, 0x8e, 0x8f, 0x90, + 0x91, 0x92, 0x93, 0x94, 0x95, 0x96, 0x97, 0x98, + 0x99, 0x9a, 0x9b, 0x9c, 0x9d, 0x9e, 0x9f, 0xa0 }, + .psize = 40, + .digest = { 0x59, 0x33, 0xe6, 0x7a } + }, + { + .key = { 0xa6, 0xcc, 0x19, 0x85 }, + .ksize = 4, + .plaintext = { 0xa1, 0xa2, 0xa3, 0xa4, 0xa5, 0xa6, 0xa7, 0xa8, + 0xa9, 0xaa, 0xab, 0xac, 0xad, 0xae, 0xaf, 0xb0, + 0xb1, 0xb2, 0xb3, 0xb4, 0xb5, 0xb6, 0xb7, 0xb8, + 0xb9, 0xba, 0xbb, 0xbc, 0xbd, 0xbe, 0xbf, 0xc0, + 0xc1, 0xc2, 0xc3, 0xc4, 0xc5, 0xc6, 0xc7, 0xc8 }, + .psize = 40, + .digest = { 0xbe, 0x03, 0x01, 0xd2 } + }, + { + .key = { 0x41, 0xfc, 0xfe, 0x2d }, + .ksize = 4, + .plaintext = { 0xc9, 0xca, 0xcb, 0xcc, 0xcd, 0xce, 0xcf, 0xd0, + 0xd1, 0xd2, 0xd3, 0xd4, 0xd5, 0xd6, 0xd7, 0xd8, + 0xd9, 0xda, 0xdb, 0xdc, 0xdd, 0xde, 0xdf, 0xe0, + 0xe1, 0xe2, 0xe3, 0xe4, 0xe5, 0xe6, 0xe7, 0xe8, + 0xe9, 0xea, 0xeb, 0xec, 0xed, 0xee, 0xef, 0xf0 }, + .psize = 40, + .digest = { 0x75, 0xd3, 0xc5, 0x24 } + }, + { + .key = { 0xff, 0xff, 0xff, 0xff }, + .ksize = 4, + .plaintext = { 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, + 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f, 0x10, + 0x11, 0x12, 0x13, 0x14, 0x15, 0x16, 0x17, 0x18, + 0x19, 0x1a, 0x1b, 0x1c, 0x1d, 0x1e, 0x1f, 0x20, + 0x21, 0x22, 0x23, 0x24, 0x25, 0x26, 0x27, 0x28, + 0x29, 0x2a, 0x2b, 0x2c, 0x2d, 0x2e, 0x2f, 0x30, + 0x31, 0x32, 0x33, 0x34, 0x35, 0x36, 0x37, 0x38, + 0x39, 0x3a, 0x3b, 0x3c, 0x3d, 0x3e, 0x3f, 0x40, + 0x41, 0x42, 0x43, 0x44, 0x45, 0x46, 0x47, 0x48, + 0x49, 0x4a, 0x4b, 0x4c, 0x4d, 0x4e, 0x4f, 0x50, + 0x51, 0x52, 0x53, 0x54, 0x55, 0x56, 0x57, 0x58, + 0x59, 0x5a, 0x5b, 0x5c, 0x5d, 0x5e, 0x5f, 0x60, + 0x61, 0x62, 0x63, 0x64, 0x65, 0x66, 0x67, 0x68, + 0x69, 0x6a, 0x6b, 0x6c, 0x6d, 0x6e, 0x6f, 0x70, + 0x71, 0x72, 0x73, 0x74, 0x75, 0x76, 0x77, 0x78, + 0x79, 0x7a, 0x7b, 0x7c, 0x7d, 0x7e, 0x7f, 0x80, + 0x81, 0x82, 0x83, 0x84, 0x85, 0x86, 0x87, 0x88, + 0x89, 0x8a, 0x8b, 0x8c, 0x8d, 0x8e, 0x8f, 0x90, + 0x91, 0x92, 0x93, 0x94, 0x95, 0x96, 0x97, 0x98, + 0x99, 0x9a, 0x9b, 0x9c, 0x9d, 0x9e, 0x9f, 0xa0, + 0xa1, 0xa2, 0xa3, 0xa4, 0xa5, 0xa6, 0xa7, 0xa8, + 0xa9, 0xaa, 0xab, 0xac, 0xad, 0xae, 0xaf, 0xb0, + 0xb1, 0xb2, 0xb3, 0xb4, 0xb5, 0xb6, 0xb7, 0xb8, + 0xb9, 0xba, 0xbb, 0xbc, 0xbd, 0xbe, 0xbf, 0xc0, + 0xc1, 0xc2, 0xc3, 0xc4, 0xc5, 0xc6, 0xc7, 0xc8, + 0xc9, 0xca, 0xcb, 0xcc, 0xcd, 0xce, 0xcf, 0xd0, + 0xd1, 0xd2, 0xd3, 0xd4, 0xd5, 0xd6, 0xd7, 0xd8, + 0xd9, 0xda, 0xdb, 0xdc, 0xdd, 0xde, 0xdf, 0xe0, + 0xe1, 0xe2, 0xe3, 0xe4, 0xe5, 0xe6, 0xe7, 0xe8, + 0xe9, 0xea, 0xeb, 0xec, 0xed, 0xee, 0xef, 0xf0 }, + .psize = 240, + .digest = { 0x75, 0xd3, 0xc5, 0x24 }, + .np = 2, + .tap = { 31, 209 } + }, +}; + +/* * Cipher speed tests */ static struct cipher_speed aes_speed_template[] = { diff --git a/crypto/tea.c b/crypto/tea.c index 5367adc..1c54e26 100644 --- a/crypto/tea.c +++ b/crypto/tea.c @@ -46,16 +46,10 @@ struct xtea_ctx { }; static int tea_setkey(struct crypto_tfm *tfm, const u8 *in_key, - unsigned int key_len, u32 *flags) + unsigned int key_len) { struct tea_ctx *ctx = crypto_tfm_ctx(tfm); const __le32 *key = (const __le32 *)in_key; - - if (key_len != 16) - { - *flags |= CRYPTO_TFM_RES_BAD_KEY_LEN; - return -EINVAL; - } ctx->KEY[0] = le32_to_cpu(key[0]); ctx->KEY[1] = le32_to_cpu(key[1]); @@ -125,16 +119,10 @@ static void tea_decrypt(struct crypto_tf } static int xtea_setkey(struct crypto_tfm *tfm, const u8 *in_key, - unsigned int key_len, u32 *flags) + unsigned int key_len) { struct xtea_ctx *ctx = crypto_tfm_ctx(tfm); const __le32 *key = (const __le32 *)in_key; - - if (key_len != 16) - { - *flags |= CRYPTO_TFM_RES_BAD_KEY_LEN; - return -EINVAL; - } ctx->KEY[0] = le32_to_cpu(key[0]); ctx->KEY[1] = le32_to_cpu(key[1]); diff --git a/crypto/twofish.c b/crypto/twofish.c index ec24882..4979a2b 100644 --- a/crypto/twofish.c +++ b/crypto/twofish.c @@ -39,6 +39,7 @@ */ #include +#include #include #include #include @@ -46,534 +47,6 @@ #include #include #include - -/* The large precomputed tables for the Twofish cipher (twofish.c) - * Taken from the same source as twofish.c - * Marc Mutz - */ - -/* These two tables are the q0 and q1 permutations, exactly as described in - * the Twofish paper. */ - -static const u8 q0[256] = { - 0xA9, 0x67, 0xB3, 0xE8, 0x04, 0xFD, 0xA3, 0x76, 0x9A, 0x92, 0x80, 0x78, - 0xE4, 0xDD, 0xD1, 0x38, 0x0D, 0xC6, 0x35, 0x98, 0x18, 0xF7, 0xEC, 0x6C, - 0x43, 0x75, 0x37, 0x26, 0xFA, 0x13, 0x94, 0x48, 0xF2, 0xD0, 0x8B, 0x30, - 0x84, 0x54, 0xDF, 0x23, 0x19, 0x5B, 0x3D, 0x59, 0xF3, 0xAE, 0xA2, 0x82, - 0x63, 0x01, 0x83, 0x2E, 0xD9, 0x51, 0x9B, 0x7C, 0xA6, 0xEB, 0xA5, 0xBE, - 0x16, 0x0C, 0xE3, 0x61, 0xC0, 0x8C, 0x3A, 0xF5, 0x73, 0x2C, 0x25, 0x0B, - 0xBB, 0x4E, 0x89, 0x6B, 0x53, 0x6A, 0xB4, 0xF1, 0xE1, 0xE6, 0xBD, 0x45, - 0xE2, 0xF4, 0xB6, 0x66, 0xCC, 0x95, 0x03, 0x56, 0xD4, 0x1C, 0x1E, 0xD7, - 0xFB, 0xC3, 0x8E, 0xB5, 0xE9, 0xCF, 0xBF, 0xBA, 0xEA, 0x77, 0x39, 0xAF, - 0x33, 0xC9, 0x62, 0x71, 0x81, 0x79, 0x09, 0xAD, 0x24, 0xCD, 0xF9, 0xD8, - 0xE5, 0xC5, 0xB9, 0x4D, 0x44, 0x08, 0x86, 0xE7, 0xA1, 0x1D, 0xAA, 0xED, - 0x06, 0x70, 0xB2, 0xD2, 0x41, 0x7B, 0xA0, 0x11, 0x31, 0xC2, 0x27, 0x90, - 0x20, 0xF6, 0x60, 0xFF, 0x96, 0x5C, 0xB1, 0xAB, 0x9E, 0x9C, 0x52, 0x1B, - 0x5F, 0x93, 0x0A, 0xEF, 0x91, 0x85, 0x49, 0xEE, 0x2D, 0x4F, 0x8F, 0x3B, - 0x47, 0x87, 0x6D, 0x46, 0xD6, 0x3E, 0x69, 0x64, 0x2A, 0xCE, 0xCB, 0x2F, - 0xFC, 0x97, 0x05, 0x7A, 0xAC, 0x7F, 0xD5, 0x1A, 0x4B, 0x0E, 0xA7, 0x5A, - 0x28, 0x14, 0x3F, 0x29, 0x88, 0x3C, 0x4C, 0x02, 0xB8, 0xDA, 0xB0, 0x17, - 0x55, 0x1F, 0x8A, 0x7D, 0x57, 0xC7, 0x8D, 0x74, 0xB7, 0xC4, 0x9F, 0x72, - 0x7E, 0x15, 0x22, 0x12, 0x58, 0x07, 0x99, 0x34, 0x6E, 0x50, 0xDE, 0x68, - 0x65, 0xBC, 0xDB, 0xF8, 0xC8, 0xA8, 0x2B, 0x40, 0xDC, 0xFE, 0x32, 0xA4, - 0xCA, 0x10, 0x21, 0xF0, 0xD3, 0x5D, 0x0F, 0x00, 0x6F, 0x9D, 0x36, 0x42, - 0x4A, 0x5E, 0xC1, 0xE0 -}; - -static const u8 q1[256] = { - 0x75, 0xF3, 0xC6, 0xF4, 0xDB, 0x7B, 0xFB, 0xC8, 0x4A, 0xD3, 0xE6, 0x6B, - 0x45, 0x7D, 0xE8, 0x4B, 0xD6, 0x32, 0xD8, 0xFD, 0x37, 0x71, 0xF1, 0xE1, - 0x30, 0x0F, 0xF8, 0x1B, 0x87, 0xFA, 0x06, 0x3F, 0x5E, 0xBA, 0xAE, 0x5B, - 0x8A, 0x00, 0xBC, 0x9D, 0x6D, 0xC1, 0xB1, 0x0E, 0x80, 0x5D, 0xD2, 0xD5, - 0xA0, 0x84, 0x07, 0x14, 0xB5, 0x90, 0x2C, 0xA3, 0xB2, 0x73, 0x4C, 0x54, - 0x92, 0x74, 0x36, 0x51, 0x38, 0xB0, 0xBD, 0x5A, 0xFC, 0x60, 0x62, 0x96, - 0x6C, 0x42, 0xF7, 0x10, 0x7C, 0x28, 0x27, 0x8C, 0x13, 0x95, 0x9C, 0xC7, - 0x24, 0x46, 0x3B, 0x70, 0xCA, 0xE3, 0x85, 0xCB, 0x11, 0xD0, 0x93, 0xB8, - 0xA6, 0x83, 0x20, 0xFF, 0x9F, 0x77, 0xC3, 0xCC, 0x03, 0x6F, 0x08, 0xBF, - 0x40, 0xE7, 0x2B, 0xE2, 0x79, 0x0C, 0xAA, 0x82, 0x41, 0x3A, 0xEA, 0xB9, - 0xE4, 0x9A, 0xA4, 0x97, 0x7E, 0xDA, 0x7A, 0x17, 0x66, 0x94, 0xA1, 0x1D, - 0x3D, 0xF0, 0xDE, 0xB3, 0x0B, 0x72, 0xA7, 0x1C, 0xEF, 0xD1, 0x53, 0x3E, - 0x8F, 0x33, 0x26, 0x5F, 0xEC, 0x76, 0x2A, 0x49, 0x81, 0x88, 0xEE, 0x21, - 0xC4, 0x1A, 0xEB, 0xD9, 0xC5, 0x39, 0x99, 0xCD, 0xAD, 0x31, 0x8B, 0x01, - 0x18, 0x23, 0xDD, 0x1F, 0x4E, 0x2D, 0xF9, 0x48, 0x4F, 0xF2, 0x65, 0x8E, - 0x78, 0x5C, 0x58, 0x19, 0x8D, 0xE5, 0x98, 0x57, 0x67, 0x7F, 0x05, 0x64, - 0xAF, 0x63, 0xB6, 0xFE, 0xF5, 0xB7, 0x3C, 0xA5, 0xCE, 0xE9, 0x68, 0x44, - 0xE0, 0x4D, 0x43, 0x69, 0x29, 0x2E, 0xAC, 0x15, 0x59, 0xA8, 0x0A, 0x9E, - 0x6E, 0x47, 0xDF, 0x34, 0x35, 0x6A, 0xCF, 0xDC, 0x22, 0xC9, 0xC0, 0x9B, - 0x89, 0xD4, 0xED, 0xAB, 0x12, 0xA2, 0x0D, 0x52, 0xBB, 0x02, 0x2F, 0xA9, - 0xD7, 0x61, 0x1E, 0xB4, 0x50, 0x04, 0xF6, 0xC2, 0x16, 0x25, 0x86, 0x56, - 0x55, 0x09, 0xBE, 0x91 -}; - -/* These MDS tables are actually tables of MDS composed with q0 and q1, - * because it is only ever used that way and we can save some time by - * precomputing. Of course the main saving comes from precomputing the - * GF(2^8) multiplication involved in the MDS matrix multiply; by looking - * things up in these tables we reduce the matrix multiply to four lookups - * and three XORs. Semi-formally, the definition of these tables is: - * mds[0][i] = MDS (q1[i] 0 0 0)^T mds[1][i] = MDS (0 q0[i] 0 0)^T - * mds[2][i] = MDS (0 0 q1[i] 0)^T mds[3][i] = MDS (0 0 0 q0[i])^T - * where ^T means "transpose", the matrix multiply is performed in GF(2^8) - * represented as GF(2)[x]/v(x) where v(x)=x^8+x^6+x^5+x^3+1 as described - * by Schneier et al, and I'm casually glossing over the byte/word - * conversion issues. */ - -static const u32 mds[4][256] = { - {0xBCBC3275, 0xECEC21F3, 0x202043C6, 0xB3B3C9F4, 0xDADA03DB, 0x02028B7B, - 0xE2E22BFB, 0x9E9EFAC8, 0xC9C9EC4A, 0xD4D409D3, 0x18186BE6, 0x1E1E9F6B, - 0x98980E45, 0xB2B2387D, 0xA6A6D2E8, 0x2626B74B, 0x3C3C57D6, 0x93938A32, - 0x8282EED8, 0x525298FD, 0x7B7BD437, 0xBBBB3771, 0x5B5B97F1, 0x474783E1, - 0x24243C30, 0x5151E20F, 0xBABAC6F8, 0x4A4AF31B, 0xBFBF4887, 0x0D0D70FA, - 0xB0B0B306, 0x7575DE3F, 0xD2D2FD5E, 0x7D7D20BA, 0x666631AE, 0x3A3AA35B, - 0x59591C8A, 0x00000000, 0xCDCD93BC, 0x1A1AE09D, 0xAEAE2C6D, 0x7F7FABC1, - 0x2B2BC7B1, 0xBEBEB90E, 0xE0E0A080, 0x8A8A105D, 0x3B3B52D2, 0x6464BAD5, - 0xD8D888A0, 0xE7E7A584, 0x5F5FE807, 0x1B1B1114, 0x2C2CC2B5, 0xFCFCB490, - 0x3131272C, 0x808065A3, 0x73732AB2, 0x0C0C8173, 0x79795F4C, 0x6B6B4154, - 0x4B4B0292, 0x53536974, 0x94948F36, 0x83831F51, 0x2A2A3638, 0xC4C49CB0, - 0x2222C8BD, 0xD5D5F85A, 0xBDBDC3FC, 0x48487860, 0xFFFFCE62, 0x4C4C0796, - 0x4141776C, 0xC7C7E642, 0xEBEB24F7, 0x1C1C1410, 0x5D5D637C, 0x36362228, - 0x6767C027, 0xE9E9AF8C, 0x4444F913, 0x1414EA95, 0xF5F5BB9C, 0xCFCF18C7, - 0x3F3F2D24, 0xC0C0E346, 0x7272DB3B, 0x54546C70, 0x29294CCA, 0xF0F035E3, - 0x0808FE85, 0xC6C617CB, 0xF3F34F11, 0x8C8CE4D0, 0xA4A45993, 0xCACA96B8, - 0x68683BA6, 0xB8B84D83, 0x38382820, 0xE5E52EFF, 0xADAD569F, 0x0B0B8477, - 0xC8C81DC3, 0x9999FFCC, 0x5858ED03, 0x19199A6F, 0x0E0E0A08, 0x95957EBF, - 0x70705040, 0xF7F730E7, 0x6E6ECF2B, 0x1F1F6EE2, 0xB5B53D79, 0x09090F0C, - 0x616134AA, 0x57571682, 0x9F9F0B41, 0x9D9D803A, 0x111164EA, 0x2525CDB9, - 0xAFAFDDE4, 0x4545089A, 0xDFDF8DA4, 0xA3A35C97, 0xEAEAD57E, 0x353558DA, - 0xEDEDD07A, 0x4343FC17, 0xF8F8CB66, 0xFBFBB194, 0x3737D3A1, 0xFAFA401D, - 0xC2C2683D, 0xB4B4CCF0, 0x32325DDE, 0x9C9C71B3, 0x5656E70B, 0xE3E3DA72, - 0x878760A7, 0x15151B1C, 0xF9F93AEF, 0x6363BFD1, 0x3434A953, 0x9A9A853E, - 0xB1B1428F, 0x7C7CD133, 0x88889B26, 0x3D3DA65F, 0xA1A1D7EC, 0xE4E4DF76, - 0x8181942A, 0x91910149, 0x0F0FFB81, 0xEEEEAA88, 0x161661EE, 0xD7D77321, - 0x9797F5C4, 0xA5A5A81A, 0xFEFE3FEB, 0x6D6DB5D9, 0x7878AEC5, 0xC5C56D39, - 0x1D1DE599, 0x7676A4CD, 0x3E3EDCAD, 0xCBCB6731, 0xB6B6478B, 0xEFEF5B01, - 0x12121E18, 0x6060C523, 0x6A6AB0DD, 0x4D4DF61F, 0xCECEE94E, 0xDEDE7C2D, - 0x55559DF9, 0x7E7E5A48, 0x2121B24F, 0x03037AF2, 0xA0A02665, 0x5E5E198E, - 0x5A5A6678, 0x65654B5C, 0x62624E58, 0xFDFD4519, 0x0606F48D, 0x404086E5, - 0xF2F2BE98, 0x3333AC57, 0x17179067, 0x05058E7F, 0xE8E85E05, 0x4F4F7D64, - 0x89896AAF, 0x10109563, 0x74742FB6, 0x0A0A75FE, 0x5C5C92F5, 0x9B9B74B7, - 0x2D2D333C, 0x3030D6A5, 0x2E2E49CE, 0x494989E9, 0x46467268, 0x77775544, - 0xA8A8D8E0, 0x9696044D, 0x2828BD43, 0xA9A92969, 0xD9D97929, 0x8686912E, - 0xD1D187AC, 0xF4F44A15, 0x8D8D1559, 0xD6D682A8, 0xB9B9BC0A, 0x42420D9E, - 0xF6F6C16E, 0x2F2FB847, 0xDDDD06DF, 0x23233934, 0xCCCC6235, 0xF1F1C46A, - 0xC1C112CF, 0x8585EBDC, 0x8F8F9E22, 0x7171A1C9, 0x9090F0C0, 0xAAAA539B, - 0x0101F189, 0x8B8BE1D4, 0x4E4E8CED, 0x8E8E6FAB, 0xABABA212, 0x6F6F3EA2, - 0xE6E6540D, 0xDBDBF252, 0x92927BBB, 0xB7B7B602, 0x6969CA2F, 0x3939D9A9, - 0xD3D30CD7, 0xA7A72361, 0xA2A2AD1E, 0xC3C399B4, 0x6C6C4450, 0x07070504, - 0x04047FF6, 0x272746C2, 0xACACA716, 0xD0D07625, 0x50501386, 0xDCDCF756, - 0x84841A55, 0xE1E15109, 0x7A7A25BE, 0x1313EF91}, - - {0xA9D93939, 0x67901717, 0xB3719C9C, 0xE8D2A6A6, 0x04050707, 0xFD985252, - 0xA3658080, 0x76DFE4E4, 0x9A084545, 0x92024B4B, 0x80A0E0E0, 0x78665A5A, - 0xE4DDAFAF, 0xDDB06A6A, 0xD1BF6363, 0x38362A2A, 0x0D54E6E6, 0xC6432020, - 0x3562CCCC, 0x98BEF2F2, 0x181E1212, 0xF724EBEB, 0xECD7A1A1, 0x6C774141, - 0x43BD2828, 0x7532BCBC, 0x37D47B7B, 0x269B8888, 0xFA700D0D, 0x13F94444, - 0x94B1FBFB, 0x485A7E7E, 0xF27A0303, 0xD0E48C8C, 0x8B47B6B6, 0x303C2424, - 0x84A5E7E7, 0x54416B6B, 0xDF06DDDD, 0x23C56060, 0x1945FDFD, 0x5BA33A3A, - 0x3D68C2C2, 0x59158D8D, 0xF321ECEC, 0xAE316666, 0xA23E6F6F, 0x82165757, - 0x63951010, 0x015BEFEF, 0x834DB8B8, 0x2E918686, 0xD9B56D6D, 0x511F8383, - 0x9B53AAAA, 0x7C635D5D, 0xA63B6868, 0xEB3FFEFE, 0xA5D63030, 0xBE257A7A, - 0x16A7ACAC, 0x0C0F0909, 0xE335F0F0, 0x6123A7A7, 0xC0F09090, 0x8CAFE9E9, - 0x3A809D9D, 0xF5925C5C, 0x73810C0C, 0x2C273131, 0x2576D0D0, 0x0BE75656, - 0xBB7B9292, 0x4EE9CECE, 0x89F10101, 0x6B9F1E1E, 0x53A93434, 0x6AC4F1F1, - 0xB499C3C3, 0xF1975B5B, 0xE1834747, 0xE66B1818, 0xBDC82222, 0x450E9898, - 0xE26E1F1F, 0xF4C9B3B3, 0xB62F7474, 0x66CBF8F8, 0xCCFF9999, 0x95EA1414, - 0x03ED5858, 0x56F7DCDC, 0xD4E18B8B, 0x1C1B1515, 0x1EADA2A2, 0xD70CD3D3, - 0xFB2BE2E2, 0xC31DC8C8, 0x8E195E5E, 0xB5C22C2C, 0xE9894949, 0xCF12C1C1, - 0xBF7E9595, 0xBA207D7D, 0xEA641111, 0x77840B0B, 0x396DC5C5, 0xAF6A8989, - 0x33D17C7C, 0xC9A17171, 0x62CEFFFF, 0x7137BBBB, 0x81FB0F0F, 0x793DB5B5, - 0x0951E1E1, 0xADDC3E3E, 0x242D3F3F, 0xCDA47676, 0xF99D5555, 0xD8EE8282, - 0xE5864040, 0xC5AE7878, 0xB9CD2525, 0x4D049696, 0x44557777, 0x080A0E0E, - 0x86135050, 0xE730F7F7, 0xA1D33737, 0x1D40FAFA, 0xAA346161, 0xED8C4E4E, - 0x06B3B0B0, 0x706C5454, 0xB22A7373, 0xD2523B3B, 0x410B9F9F, 0x7B8B0202, - 0xA088D8D8, 0x114FF3F3, 0x3167CBCB, 0xC2462727, 0x27C06767, 0x90B4FCFC, - 0x20283838, 0xF67F0404, 0x60784848, 0xFF2EE5E5, 0x96074C4C, 0x5C4B6565, - 0xB1C72B2B, 0xAB6F8E8E, 0x9E0D4242, 0x9CBBF5F5, 0x52F2DBDB, 0x1BF34A4A, - 0x5FA63D3D, 0x9359A4A4, 0x0ABCB9B9, 0xEF3AF9F9, 0x91EF1313, 0x85FE0808, - 0x49019191, 0xEE611616, 0x2D7CDEDE, 0x4FB22121, 0x8F42B1B1, 0x3BDB7272, - 0x47B82F2F, 0x8748BFBF, 0x6D2CAEAE, 0x46E3C0C0, 0xD6573C3C, 0x3E859A9A, - 0x6929A9A9, 0x647D4F4F, 0x2A948181, 0xCE492E2E, 0xCB17C6C6, 0x2FCA6969, - 0xFCC3BDBD, 0x975CA3A3, 0x055EE8E8, 0x7AD0EDED, 0xAC87D1D1, 0x7F8E0505, - 0xD5BA6464, 0x1AA8A5A5, 0x4BB72626, 0x0EB9BEBE, 0xA7608787, 0x5AF8D5D5, - 0x28223636, 0x14111B1B, 0x3FDE7575, 0x2979D9D9, 0x88AAEEEE, 0x3C332D2D, - 0x4C5F7979, 0x02B6B7B7, 0xB896CACA, 0xDA583535, 0xB09CC4C4, 0x17FC4343, - 0x551A8484, 0x1FF64D4D, 0x8A1C5959, 0x7D38B2B2, 0x57AC3333, 0xC718CFCF, - 0x8DF40606, 0x74695353, 0xB7749B9B, 0xC4F59797, 0x9F56ADAD, 0x72DAE3E3, - 0x7ED5EAEA, 0x154AF4F4, 0x229E8F8F, 0x12A2ABAB, 0x584E6262, 0x07E85F5F, - 0x99E51D1D, 0x34392323, 0x6EC1F6F6, 0x50446C6C, 0xDE5D3232, 0x68724646, - 0x6526A0A0, 0xBC93CDCD, 0xDB03DADA, 0xF8C6BABA, 0xC8FA9E9E, 0xA882D6D6, - 0x2BCF6E6E, 0x40507070, 0xDCEB8585, 0xFE750A0A, 0x328A9393, 0xA48DDFDF, - 0xCA4C2929, 0x10141C1C, 0x2173D7D7, 0xF0CCB4B4, 0xD309D4D4, 0x5D108A8A, - 0x0FE25151, 0x00000000, 0x6F9A1919, 0x9DE01A1A, 0x368F9494, 0x42E6C7C7, - 0x4AECC9C9, 0x5EFDD2D2, 0xC1AB7F7F, 0xE0D8A8A8}, - - {0xBC75BC32, 0xECF3EC21, 0x20C62043, 0xB3F4B3C9, 0xDADBDA03, 0x027B028B, - 0xE2FBE22B, 0x9EC89EFA, 0xC94AC9EC, 0xD4D3D409, 0x18E6186B, 0x1E6B1E9F, - 0x9845980E, 0xB27DB238, 0xA6E8A6D2, 0x264B26B7, 0x3CD63C57, 0x9332938A, - 0x82D882EE, 0x52FD5298, 0x7B377BD4, 0xBB71BB37, 0x5BF15B97, 0x47E14783, - 0x2430243C, 0x510F51E2, 0xBAF8BAC6, 0x4A1B4AF3, 0xBF87BF48, 0x0DFA0D70, - 0xB006B0B3, 0x753F75DE, 0xD25ED2FD, 0x7DBA7D20, 0x66AE6631, 0x3A5B3AA3, - 0x598A591C, 0x00000000, 0xCDBCCD93, 0x1A9D1AE0, 0xAE6DAE2C, 0x7FC17FAB, - 0x2BB12BC7, 0xBE0EBEB9, 0xE080E0A0, 0x8A5D8A10, 0x3BD23B52, 0x64D564BA, - 0xD8A0D888, 0xE784E7A5, 0x5F075FE8, 0x1B141B11, 0x2CB52CC2, 0xFC90FCB4, - 0x312C3127, 0x80A38065, 0x73B2732A, 0x0C730C81, 0x794C795F, 0x6B546B41, - 0x4B924B02, 0x53745369, 0x9436948F, 0x8351831F, 0x2A382A36, 0xC4B0C49C, - 0x22BD22C8, 0xD55AD5F8, 0xBDFCBDC3, 0x48604878, 0xFF62FFCE, 0x4C964C07, - 0x416C4177, 0xC742C7E6, 0xEBF7EB24, 0x1C101C14, 0x5D7C5D63, 0x36283622, - 0x672767C0, 0xE98CE9AF, 0x441344F9, 0x149514EA, 0xF59CF5BB, 0xCFC7CF18, - 0x3F243F2D, 0xC046C0E3, 0x723B72DB, 0x5470546C, 0x29CA294C, 0xF0E3F035, - 0x088508FE, 0xC6CBC617, 0xF311F34F, 0x8CD08CE4, 0xA493A459, 0xCAB8CA96, - 0x68A6683B, 0xB883B84D, 0x38203828, 0xE5FFE52E, 0xAD9FAD56, 0x0B770B84, - 0xC8C3C81D, 0x99CC99FF, 0x580358ED, 0x196F199A, 0x0E080E0A, 0x95BF957E, - 0x70407050, 0xF7E7F730, 0x6E2B6ECF, 0x1FE21F6E, 0xB579B53D, 0x090C090F, - 0x61AA6134, 0x57825716, 0x9F419F0B, 0x9D3A9D80, 0x11EA1164, 0x25B925CD, - 0xAFE4AFDD, 0x459A4508, 0xDFA4DF8D, 0xA397A35C, 0xEA7EEAD5, 0x35DA3558, - 0xED7AEDD0, 0x431743FC, 0xF866F8CB, 0xFB94FBB1, 0x37A137D3, 0xFA1DFA40, - 0xC23DC268, 0xB4F0B4CC, 0x32DE325D, 0x9CB39C71, 0x560B56E7, 0xE372E3DA, - 0x87A78760, 0x151C151B, 0xF9EFF93A, 0x63D163BF, 0x345334A9, 0x9A3E9A85, - 0xB18FB142, 0x7C337CD1, 0x8826889B, 0x3D5F3DA6, 0xA1ECA1D7, 0xE476E4DF, - 0x812A8194, 0x91499101, 0x0F810FFB, 0xEE88EEAA, 0x16EE1661, 0xD721D773, - 0x97C497F5, 0xA51AA5A8, 0xFEEBFE3F, 0x6DD96DB5, 0x78C578AE, 0xC539C56D, - 0x1D991DE5, 0x76CD76A4, 0x3EAD3EDC, 0xCB31CB67, 0xB68BB647, 0xEF01EF5B, - 0x1218121E, 0x602360C5, 0x6ADD6AB0, 0x4D1F4DF6, 0xCE4ECEE9, 0xDE2DDE7C, - 0x55F9559D, 0x7E487E5A, 0x214F21B2, 0x03F2037A, 0xA065A026, 0x5E8E5E19, - 0x5A785A66, 0x655C654B, 0x6258624E, 0xFD19FD45, 0x068D06F4, 0x40E54086, - 0xF298F2BE, 0x335733AC, 0x17671790, 0x057F058E, 0xE805E85E, 0x4F644F7D, - 0x89AF896A, 0x10631095, 0x74B6742F, 0x0AFE0A75, 0x5CF55C92, 0x9BB79B74, - 0x2D3C2D33, 0x30A530D6, 0x2ECE2E49, 0x49E94989, 0x46684672, 0x77447755, - 0xA8E0A8D8, 0x964D9604, 0x284328BD, 0xA969A929, 0xD929D979, 0x862E8691, - 0xD1ACD187, 0xF415F44A, 0x8D598D15, 0xD6A8D682, 0xB90AB9BC, 0x429E420D, - 0xF66EF6C1, 0x2F472FB8, 0xDDDFDD06, 0x23342339, 0xCC35CC62, 0xF16AF1C4, - 0xC1CFC112, 0x85DC85EB, 0x8F228F9E, 0x71C971A1, 0x90C090F0, 0xAA9BAA53, - 0x018901F1, 0x8BD48BE1, 0x4EED4E8C, 0x8EAB8E6F, 0xAB12ABA2, 0x6FA26F3E, - 0xE60DE654, 0xDB52DBF2, 0x92BB927B, 0xB702B7B6, 0x692F69CA, 0x39A939D9, - 0xD3D7D30C, 0xA761A723, 0xA21EA2AD, 0xC3B4C399, 0x6C506C44, 0x07040705, - 0x04F6047F, 0x27C22746, 0xAC16ACA7, 0xD025D076, 0x50865013, 0xDC56DCF7, - 0x8455841A, 0xE109E151, 0x7ABE7A25, 0x139113EF}, - - {0xD939A9D9, 0x90176790, 0x719CB371, 0xD2A6E8D2, 0x05070405, 0x9852FD98, - 0x6580A365, 0xDFE476DF, 0x08459A08, 0x024B9202, 0xA0E080A0, 0x665A7866, - 0xDDAFE4DD, 0xB06ADDB0, 0xBF63D1BF, 0x362A3836, 0x54E60D54, 0x4320C643, - 0x62CC3562, 0xBEF298BE, 0x1E12181E, 0x24EBF724, 0xD7A1ECD7, 0x77416C77, - 0xBD2843BD, 0x32BC7532, 0xD47B37D4, 0x9B88269B, 0x700DFA70, 0xF94413F9, - 0xB1FB94B1, 0x5A7E485A, 0x7A03F27A, 0xE48CD0E4, 0x47B68B47, 0x3C24303C, - 0xA5E784A5, 0x416B5441, 0x06DDDF06, 0xC56023C5, 0x45FD1945, 0xA33A5BA3, - 0x68C23D68, 0x158D5915, 0x21ECF321, 0x3166AE31, 0x3E6FA23E, 0x16578216, - 0x95106395, 0x5BEF015B, 0x4DB8834D, 0x91862E91, 0xB56DD9B5, 0x1F83511F, - 0x53AA9B53, 0x635D7C63, 0x3B68A63B, 0x3FFEEB3F, 0xD630A5D6, 0x257ABE25, - 0xA7AC16A7, 0x0F090C0F, 0x35F0E335, 0x23A76123, 0xF090C0F0, 0xAFE98CAF, - 0x809D3A80, 0x925CF592, 0x810C7381, 0x27312C27, 0x76D02576, 0xE7560BE7, - 0x7B92BB7B, 0xE9CE4EE9, 0xF10189F1, 0x9F1E6B9F, 0xA93453A9, 0xC4F16AC4, - 0x99C3B499, 0x975BF197, 0x8347E183, 0x6B18E66B, 0xC822BDC8, 0x0E98450E, - 0x6E1FE26E, 0xC9B3F4C9, 0x2F74B62F, 0xCBF866CB, 0xFF99CCFF, 0xEA1495EA, - 0xED5803ED, 0xF7DC56F7, 0xE18BD4E1, 0x1B151C1B, 0xADA21EAD, 0x0CD3D70C, - 0x2BE2FB2B, 0x1DC8C31D, 0x195E8E19, 0xC22CB5C2, 0x8949E989, 0x12C1CF12, - 0x7E95BF7E, 0x207DBA20, 0x6411EA64, 0x840B7784, 0x6DC5396D, 0x6A89AF6A, - 0xD17C33D1, 0xA171C9A1, 0xCEFF62CE, 0x37BB7137, 0xFB0F81FB, 0x3DB5793D, - 0x51E10951, 0xDC3EADDC, 0x2D3F242D, 0xA476CDA4, 0x9D55F99D, 0xEE82D8EE, - 0x8640E586, 0xAE78C5AE, 0xCD25B9CD, 0x04964D04, 0x55774455, 0x0A0E080A, - 0x13508613, 0x30F7E730, 0xD337A1D3, 0x40FA1D40, 0x3461AA34, 0x8C4EED8C, - 0xB3B006B3, 0x6C54706C, 0x2A73B22A, 0x523BD252, 0x0B9F410B, 0x8B027B8B, - 0x88D8A088, 0x4FF3114F, 0x67CB3167, 0x4627C246, 0xC06727C0, 0xB4FC90B4, - 0x28382028, 0x7F04F67F, 0x78486078, 0x2EE5FF2E, 0x074C9607, 0x4B655C4B, - 0xC72BB1C7, 0x6F8EAB6F, 0x0D429E0D, 0xBBF59CBB, 0xF2DB52F2, 0xF34A1BF3, - 0xA63D5FA6, 0x59A49359, 0xBCB90ABC, 0x3AF9EF3A, 0xEF1391EF, 0xFE0885FE, - 0x01914901, 0x6116EE61, 0x7CDE2D7C, 0xB2214FB2, 0x42B18F42, 0xDB723BDB, - 0xB82F47B8, 0x48BF8748, 0x2CAE6D2C, 0xE3C046E3, 0x573CD657, 0x859A3E85, - 0x29A96929, 0x7D4F647D, 0x94812A94, 0x492ECE49, 0x17C6CB17, 0xCA692FCA, - 0xC3BDFCC3, 0x5CA3975C, 0x5EE8055E, 0xD0ED7AD0, 0x87D1AC87, 0x8E057F8E, - 0xBA64D5BA, 0xA8A51AA8, 0xB7264BB7, 0xB9BE0EB9, 0x6087A760, 0xF8D55AF8, - 0x22362822, 0x111B1411, 0xDE753FDE, 0x79D92979, 0xAAEE88AA, 0x332D3C33, - 0x5F794C5F, 0xB6B702B6, 0x96CAB896, 0x5835DA58, 0x9CC4B09C, 0xFC4317FC, - 0x1A84551A, 0xF64D1FF6, 0x1C598A1C, 0x38B27D38, 0xAC3357AC, 0x18CFC718, - 0xF4068DF4, 0x69537469, 0x749BB774, 0xF597C4F5, 0x56AD9F56, 0xDAE372DA, - 0xD5EA7ED5, 0x4AF4154A, 0x9E8F229E, 0xA2AB12A2, 0x4E62584E, 0xE85F07E8, - 0xE51D99E5, 0x39233439, 0xC1F66EC1, 0x446C5044, 0x5D32DE5D, 0x72466872, - 0x26A06526, 0x93CDBC93, 0x03DADB03, 0xC6BAF8C6, 0xFA9EC8FA, 0x82D6A882, - 0xCF6E2BCF, 0x50704050, 0xEB85DCEB, 0x750AFE75, 0x8A93328A, 0x8DDFA48D, - 0x4C29CA4C, 0x141C1014, 0x73D72173, 0xCCB4F0CC, 0x09D4D309, 0x108A5D10, - 0xE2510FE2, 0x00000000, 0x9A196F9A, 0xE01A9DE0, 0x8F94368F, 0xE6C742E6, - 0xECC94AEC, 0xFDD25EFD, 0xAB7FC1AB, 0xD8A8E0D8} -}; - -/* The exp_to_poly and poly_to_exp tables are used to perform efficient - * operations in GF(2^8) represented as GF(2)[x]/w(x) where - * w(x)=x^8+x^6+x^3+x^2+1. We care about doing that because it's part of the - * definition of the RS matrix in the key schedule. Elements of that field - * are polynomials of degree not greater than 7 and all coefficients 0 or 1, - * which can be represented naturally by bytes (just substitute x=2). In that - * form, GF(2^8) addition is the same as bitwise XOR, but GF(2^8) - * multiplication is inefficient without hardware support. To multiply - * faster, I make use of the fact x is a generator for the nonzero elements, - * so that every element p of GF(2)[x]/w(x) is either 0 or equal to (x)^n for - * some n in 0..254. Note that that caret is exponentiation in GF(2^8), - * *not* polynomial notation. So if I want to compute pq where p and q are - * in GF(2^8), I can just say: - * 1. if p=0 or q=0 then pq=0 - * 2. otherwise, find m and n such that p=x^m and q=x^n - * 3. pq=(x^m)(x^n)=x^(m+n), so add m and n and find pq - * The translations in steps 2 and 3 are looked up in the tables - * poly_to_exp (for step 2) and exp_to_poly (for step 3). To see this - * in action, look at the CALC_S macro. As additional wrinkles, note that - * one of my operands is always a constant, so the poly_to_exp lookup on it - * is done in advance; I included the original values in the comments so - * readers can have some chance of recognizing that this *is* the RS matrix - * from the Twofish paper. I've only included the table entries I actually - * need; I never do a lookup on a variable input of zero and the biggest - * exponents I'll ever see are 254 (variable) and 237 (constant), so they'll - * never sum to more than 491. I'm repeating part of the exp_to_poly table - * so that I don't have to do mod-255 reduction in the exponent arithmetic. - * Since I know my constant operands are never zero, I only have to worry - * about zero values in the variable operand, and I do it with a simple - * conditional branch. I know conditionals are expensive, but I couldn't - * see a non-horrible way of avoiding them, and I did manage to group the - * statements so that each if covers four group multiplications. */ - -static const u8 poly_to_exp[255] = { - 0x00, 0x01, 0x17, 0x02, 0x2E, 0x18, 0x53, 0x03, 0x6A, 0x2F, 0x93, 0x19, - 0x34, 0x54, 0x45, 0x04, 0x5C, 0x6B, 0xB6, 0x30, 0xA6, 0x94, 0x4B, 0x1A, - 0x8C, 0x35, 0x81, 0x55, 0xAA, 0x46, 0x0D, 0x05, 0x24, 0x5D, 0x87, 0x6C, - 0x9B, 0xB7, 0xC1, 0x31, 0x2B, 0xA7, 0xA3, 0x95, 0x98, 0x4C, 0xCA, 0x1B, - 0xE6, 0x8D, 0x73, 0x36, 0xCD, 0x82, 0x12, 0x56, 0x62, 0xAB, 0xF0, 0x47, - 0x4F, 0x0E, 0xBD, 0x06, 0xD4, 0x25, 0xD2, 0x5E, 0x27, 0x88, 0x66, 0x6D, - 0xD6, 0x9C, 0x79, 0xB8, 0x08, 0xC2, 0xDF, 0x32, 0x68, 0x2C, 0xFD, 0xA8, - 0x8A, 0xA4, 0x5A, 0x96, 0x29, 0x99, 0x22, 0x4D, 0x60, 0xCB, 0xE4, 0x1C, - 0x7B, 0xE7, 0x3B, 0x8E, 0x9E, 0x74, 0xF4, 0x37, 0xD8, 0xCE, 0xF9, 0x83, - 0x6F, 0x13, 0xB2, 0x57, 0xE1, 0x63, 0xDC, 0xAC, 0xC4, 0xF1, 0xAF, 0x48, - 0x0A, 0x50, 0x42, 0x0F, 0xBA, 0xBE, 0xC7, 0x07, 0xDE, 0xD5, 0x78, 0x26, - 0x65, 0xD3, 0xD1, 0x5F, 0xE3, 0x28, 0x21, 0x89, 0x59, 0x67, 0xFC, 0x6E, - 0xB1, 0xD7, 0xF8, 0x9D, 0xF3, 0x7A, 0x3A, 0xB9, 0xC6, 0x09, 0x41, 0xC3, - 0xAE, 0xE0, 0xDB, 0x33, 0x44, 0x69, 0x92, 0x2D, 0x52, 0xFE, 0x16, 0xA9, - 0x0C, 0x8B, 0x80, 0xA5, 0x4A, 0x5B, 0xB5, 0x97, 0xC9, 0x2A, 0xA2, 0x9A, - 0xC0, 0x23, 0x86, 0x4E, 0xBC, 0x61, 0xEF, 0xCC, 0x11, 0xE5, 0x72, 0x1D, - 0x3D, 0x7C, 0xEB, 0xE8, 0xE9, 0x3C, 0xEA, 0x8F, 0x7D, 0x9F, 0xEC, 0x75, - 0x1E, 0xF5, 0x3E, 0x38, 0xF6, 0xD9, 0x3F, 0xCF, 0x76, 0xFA, 0x1F, 0x84, - 0xA0, 0x70, 0xED, 0x14, 0x90, 0xB3, 0x7E, 0x58, 0xFB, 0xE2, 0x20, 0x64, - 0xD0, 0xDD, 0x77, 0xAD, 0xDA, 0xC5, 0x40, 0xF2, 0x39, 0xB0, 0xF7, 0x49, - 0xB4, 0x0B, 0x7F, 0x51, 0x15, 0x43, 0x91, 0x10, 0x71, 0xBB, 0xEE, 0xBF, - 0x85, 0xC8, 0xA1 -}; - -static const u8 exp_to_poly[492] = { - 0x01, 0x02, 0x04, 0x08, 0x10, 0x20, 0x40, 0x80, 0x4D, 0x9A, 0x79, 0xF2, - 0xA9, 0x1F, 0x3E, 0x7C, 0xF8, 0xBD, 0x37, 0x6E, 0xDC, 0xF5, 0xA7, 0x03, - 0x06, 0x0C, 0x18, 0x30, 0x60, 0xC0, 0xCD, 0xD7, 0xE3, 0x8B, 0x5B, 0xB6, - 0x21, 0x42, 0x84, 0x45, 0x8A, 0x59, 0xB2, 0x29, 0x52, 0xA4, 0x05, 0x0A, - 0x14, 0x28, 0x50, 0xA0, 0x0D, 0x1A, 0x34, 0x68, 0xD0, 0xED, 0x97, 0x63, - 0xC6, 0xC1, 0xCF, 0xD3, 0xEB, 0x9B, 0x7B, 0xF6, 0xA1, 0x0F, 0x1E, 0x3C, - 0x78, 0xF0, 0xAD, 0x17, 0x2E, 0x5C, 0xB8, 0x3D, 0x7A, 0xF4, 0xA5, 0x07, - 0x0E, 0x1C, 0x38, 0x70, 0xE0, 0x8D, 0x57, 0xAE, 0x11, 0x22, 0x44, 0x88, - 0x5D, 0xBA, 0x39, 0x72, 0xE4, 0x85, 0x47, 0x8E, 0x51, 0xA2, 0x09, 0x12, - 0x24, 0x48, 0x90, 0x6D, 0xDA, 0xF9, 0xBF, 0x33, 0x66, 0xCC, 0xD5, 0xE7, - 0x83, 0x4B, 0x96, 0x61, 0xC2, 0xC9, 0xDF, 0xF3, 0xAB, 0x1B, 0x36, 0x6C, - 0xD8, 0xFD, 0xB7, 0x23, 0x46, 0x8C, 0x55, 0xAA, 0x19, 0x32, 0x64, 0xC8, - 0xDD, 0xF7, 0xA3, 0x0B, 0x16, 0x2C, 0x58, 0xB0, 0x2D, 0x5A, 0xB4, 0x25, - 0x4A, 0x94, 0x65, 0xCA, 0xD9, 0xFF, 0xB3, 0x2B, 0x56, 0xAC, 0x15, 0x2A, - 0x54, 0xA8, 0x1D, 0x3A, 0x74, 0xE8, 0x9D, 0x77, 0xEE, 0x91, 0x6F, 0xDE, - 0xF1, 0xAF, 0x13, 0x26, 0x4C, 0x98, 0x7D, 0xFA, 0xB9, 0x3F, 0x7E, 0xFC, - 0xB5, 0x27, 0x4E, 0x9C, 0x75, 0xEA, 0x99, 0x7F, 0xFE, 0xB1, 0x2F, 0x5E, - 0xBC, 0x35, 0x6A, 0xD4, 0xE5, 0x87, 0x43, 0x86, 0x41, 0x82, 0x49, 0x92, - 0x69, 0xD2, 0xE9, 0x9F, 0x73, 0xE6, 0x81, 0x4F, 0x9E, 0x71, 0xE2, 0x89, - 0x5F, 0xBE, 0x31, 0x62, 0xC4, 0xC5, 0xC7, 0xC3, 0xCB, 0xDB, 0xFB, 0xBB, - 0x3B, 0x76, 0xEC, 0x95, 0x67, 0xCE, 0xD1, 0xEF, 0x93, 0x6B, 0xD6, 0xE1, - 0x8F, 0x53, 0xA6, 0x01, 0x02, 0x04, 0x08, 0x10, 0x20, 0x40, 0x80, 0x4D, - 0x9A, 0x79, 0xF2, 0xA9, 0x1F, 0x3E, 0x7C, 0xF8, 0xBD, 0x37, 0x6E, 0xDC, - 0xF5, 0xA7, 0x03, 0x06, 0x0C, 0x18, 0x30, 0x60, 0xC0, 0xCD, 0xD7, 0xE3, - 0x8B, 0x5B, 0xB6, 0x21, 0x42, 0x84, 0x45, 0x8A, 0x59, 0xB2, 0x29, 0x52, - 0xA4, 0x05, 0x0A, 0x14, 0x28, 0x50, 0xA0, 0x0D, 0x1A, 0x34, 0x68, 0xD0, - 0xED, 0x97, 0x63, 0xC6, 0xC1, 0xCF, 0xD3, 0xEB, 0x9B, 0x7B, 0xF6, 0xA1, - 0x0F, 0x1E, 0x3C, 0x78, 0xF0, 0xAD, 0x17, 0x2E, 0x5C, 0xB8, 0x3D, 0x7A, - 0xF4, 0xA5, 0x07, 0x0E, 0x1C, 0x38, 0x70, 0xE0, 0x8D, 0x57, 0xAE, 0x11, - 0x22, 0x44, 0x88, 0x5D, 0xBA, 0x39, 0x72, 0xE4, 0x85, 0x47, 0x8E, 0x51, - 0xA2, 0x09, 0x12, 0x24, 0x48, 0x90, 0x6D, 0xDA, 0xF9, 0xBF, 0x33, 0x66, - 0xCC, 0xD5, 0xE7, 0x83, 0x4B, 0x96, 0x61, 0xC2, 0xC9, 0xDF, 0xF3, 0xAB, - 0x1B, 0x36, 0x6C, 0xD8, 0xFD, 0xB7, 0x23, 0x46, 0x8C, 0x55, 0xAA, 0x19, - 0x32, 0x64, 0xC8, 0xDD, 0xF7, 0xA3, 0x0B, 0x16, 0x2C, 0x58, 0xB0, 0x2D, - 0x5A, 0xB4, 0x25, 0x4A, 0x94, 0x65, 0xCA, 0xD9, 0xFF, 0xB3, 0x2B, 0x56, - 0xAC, 0x15, 0x2A, 0x54, 0xA8, 0x1D, 0x3A, 0x74, 0xE8, 0x9D, 0x77, 0xEE, - 0x91, 0x6F, 0xDE, 0xF1, 0xAF, 0x13, 0x26, 0x4C, 0x98, 0x7D, 0xFA, 0xB9, - 0x3F, 0x7E, 0xFC, 0xB5, 0x27, 0x4E, 0x9C, 0x75, 0xEA, 0x99, 0x7F, 0xFE, - 0xB1, 0x2F, 0x5E, 0xBC, 0x35, 0x6A, 0xD4, 0xE5, 0x87, 0x43, 0x86, 0x41, - 0x82, 0x49, 0x92, 0x69, 0xD2, 0xE9, 0x9F, 0x73, 0xE6, 0x81, 0x4F, 0x9E, - 0x71, 0xE2, 0x89, 0x5F, 0xBE, 0x31, 0x62, 0xC4, 0xC5, 0xC7, 0xC3, 0xCB -}; - - -/* The table constants are indices of - * S-box entries, preprocessed through q0 and q1. */ -static const u8 calc_sb_tbl[512] = { - 0xA9, 0x75, 0x67, 0xF3, 0xB3, 0xC6, 0xE8, 0xF4, - 0x04, 0xDB, 0xFD, 0x7B, 0xA3, 0xFB, 0x76, 0xC8, - 0x9A, 0x4A, 0x92, 0xD3, 0x80, 0xE6, 0x78, 0x6B, - 0xE4, 0x45, 0xDD, 0x7D, 0xD1, 0xE8, 0x38, 0x4B, - 0x0D, 0xD6, 0xC6, 0x32, 0x35, 0xD8, 0x98, 0xFD, - 0x18, 0x37, 0xF7, 0x71, 0xEC, 0xF1, 0x6C, 0xE1, - 0x43, 0x30, 0x75, 0x0F, 0x37, 0xF8, 0x26, 0x1B, - 0xFA, 0x87, 0x13, 0xFA, 0x94, 0x06, 0x48, 0x3F, - 0xF2, 0x5E, 0xD0, 0xBA, 0x8B, 0xAE, 0x30, 0x5B, - 0x84, 0x8A, 0x54, 0x00, 0xDF, 0xBC, 0x23, 0x9D, - 0x19, 0x6D, 0x5B, 0xC1, 0x3D, 0xB1, 0x59, 0x0E, - 0xF3, 0x80, 0xAE, 0x5D, 0xA2, 0xD2, 0x82, 0xD5, - 0x63, 0xA0, 0x01, 0x84, 0x83, 0x07, 0x2E, 0x14, - 0xD9, 0xB5, 0x51, 0x90, 0x9B, 0x2C, 0x7C, 0xA3, - 0xA6, 0xB2, 0xEB, 0x73, 0xA5, 0x4C, 0xBE, 0x54, - 0x16, 0x92, 0x0C, 0x74, 0xE3, 0x36, 0x61, 0x51, - 0xC0, 0x38, 0x8C, 0xB0, 0x3A, 0xBD, 0xF5, 0x5A, - 0x73, 0xFC, 0x2C, 0x60, 0x25, 0x62, 0x0B, 0x96, - 0xBB, 0x6C, 0x4E, 0x42, 0x89, 0xF7, 0x6B, 0x10, - 0x53, 0x7C, 0x6A, 0x28, 0xB4, 0x27, 0xF1, 0x8C, - 0xE1, 0x13, 0xE6, 0x95, 0xBD, 0x9C, 0x45, 0xC7, - 0xE2, 0x24, 0xF4, 0x46, 0xB6, 0x3B, 0x66, 0x70, - 0xCC, 0xCA, 0x95, 0xE3, 0x03, 0x85, 0x56, 0xCB, - 0xD4, 0x11, 0x1C, 0xD0, 0x1E, 0x93, 0xD7, 0xB8, - 0xFB, 0xA6, 0xC3, 0x83, 0x8E, 0x20, 0xB5, 0xFF, - 0xE9, 0x9F, 0xCF, 0x77, 0xBF, 0xC3, 0xBA, 0xCC, - 0xEA, 0x03, 0x77, 0x6F, 0x39, 0x08, 0xAF, 0xBF, - 0x33, 0x40, 0xC9, 0xE7, 0x62, 0x2B, 0x71, 0xE2, - 0x81, 0x79, 0x79, 0x0C, 0x09, 0xAA, 0xAD, 0x82, - 0x24, 0x41, 0xCD, 0x3A, 0xF9, 0xEA, 0xD8, 0xB9, - 0xE5, 0xE4, 0xC5, 0x9A, 0xB9, 0xA4, 0x4D, 0x97, - 0x44, 0x7E, 0x08, 0xDA, 0x86, 0x7A, 0xE7, 0x17, - 0xA1, 0x66, 0x1D, 0x94, 0xAA, 0xA1, 0xED, 0x1D, - 0x06, 0x3D, 0x70, 0xF0, 0xB2, 0xDE, 0xD2, 0xB3, - 0x41, 0x0B, 0x7B, 0x72, 0xA0, 0xA7, 0x11, 0x1C, - 0x31, 0xEF, 0xC2, 0xD1, 0x27, 0x53, 0x90, 0x3E, - 0x20, 0x8F, 0xF6, 0x33, 0x60, 0x26, 0xFF, 0x5F, - 0x96, 0xEC, 0x5C, 0x76, 0xB1, 0x2A, 0xAB, 0x49, - 0x9E, 0x81, 0x9C, 0x88, 0x52, 0xEE, 0x1B, 0x21, - 0x5F, 0xC4, 0x93, 0x1A, 0x0A, 0xEB, 0xEF, 0xD9, - 0x91, 0xC5, 0x85, 0x39, 0x49, 0x99, 0xEE, 0xCD, - 0x2D, 0xAD, 0x4F, 0x31, 0x8F, 0x8B, 0x3B, 0x01, - 0x47, 0x18, 0x87, 0x23, 0x6D, 0xDD, 0x46, 0x1F, - 0xD6, 0x4E, 0x3E, 0x2D, 0x69, 0xF9, 0x64, 0x48, - 0x2A, 0x4F, 0xCE, 0xF2, 0xCB, 0x65, 0x2F, 0x8E, - 0xFC, 0x78, 0x97, 0x5C, 0x05, 0x58, 0x7A, 0x19, - 0xAC, 0x8D, 0x7F, 0xE5, 0xD5, 0x98, 0x1A, 0x57, - 0x4B, 0x67, 0x0E, 0x7F, 0xA7, 0x05, 0x5A, 0x64, - 0x28, 0xAF, 0x14, 0x63, 0x3F, 0xB6, 0x29, 0xFE, - 0x88, 0xF5, 0x3C, 0xB7, 0x4C, 0x3C, 0x02, 0xA5, - 0xB8, 0xCE, 0xDA, 0xE9, 0xB0, 0x68, 0x17, 0x44, - 0x55, 0xE0, 0x1F, 0x4D, 0x8A, 0x43, 0x7D, 0x69, - 0x57, 0x29, 0xC7, 0x2E, 0x8D, 0xAC, 0x74, 0x15, - 0xB7, 0x59, 0xC4, 0xA8, 0x9F, 0x0A, 0x72, 0x9E, - 0x7E, 0x6E, 0x15, 0x47, 0x22, 0xDF, 0x12, 0x34, - 0x58, 0x35, 0x07, 0x6A, 0x99, 0xCF, 0x34, 0xDC, - 0x6E, 0x22, 0x50, 0xC9, 0xDE, 0xC0, 0x68, 0x9B, - 0x65, 0x89, 0xBC, 0xD4, 0xDB, 0xED, 0xF8, 0xAB, - 0xC8, 0x12, 0xA8, 0xA2, 0x2B, 0x0D, 0x40, 0x52, - 0xDC, 0xBB, 0xFE, 0x02, 0x32, 0x2F, 0xA4, 0xA9, - 0xCA, 0xD7, 0x10, 0x61, 0x21, 0x1E, 0xF0, 0xB4, - 0xD3, 0x50, 0x5D, 0x04, 0x0F, 0xF6, 0x00, 0xC2, - 0x6F, 0x16, 0x9D, 0x25, 0x36, 0x86, 0x42, 0x56, - 0x4A, 0x55, 0x5E, 0x09, 0xC1, 0xBE, 0xE0, 0x91 -}; - -/* Macro to perform one column of the RS matrix multiplication. The - * parameters a, b, c, and d are the four bytes of output; i is the index - * of the key bytes, and w, x, y, and z, are the column of constants from - * the RS matrix, preprocessed through the poly_to_exp table. */ - -#define CALC_S(a, b, c, d, i, w, x, y, z) \ - if (key[i]) { \ - tmp = poly_to_exp[key[i] - 1]; \ - (a) ^= exp_to_poly[tmp + (w)]; \ - (b) ^= exp_to_poly[tmp + (x)]; \ - (c) ^= exp_to_poly[tmp + (y)]; \ - (d) ^= exp_to_poly[tmp + (z)]; \ - } - -/* Macros to calculate the key-dependent S-boxes for a 128-bit key using - * the S vector from CALC_S. CALC_SB_2 computes a single entry in all - * four S-boxes, where i is the index of the entry to compute, and a and b - * are the index numbers preprocessed through the q0 and q1 tables - * respectively. */ - -#define CALC_SB_2(i, a, b) \ - ctx->s[0][i] = mds[0][q0[(a) ^ sa] ^ se]; \ - ctx->s[1][i] = mds[1][q0[(b) ^ sb] ^ sf]; \ - ctx->s[2][i] = mds[2][q1[(a) ^ sc] ^ sg]; \ - ctx->s[3][i] = mds[3][q1[(b) ^ sd] ^ sh] - -/* Macro exactly like CALC_SB_2, but for 192-bit keys. */ - -#define CALC_SB192_2(i, a, b) \ - ctx->s[0][i] = mds[0][q0[q0[(b) ^ sa] ^ se] ^ si]; \ - ctx->s[1][i] = mds[1][q0[q1[(b) ^ sb] ^ sf] ^ sj]; \ - ctx->s[2][i] = mds[2][q1[q0[(a) ^ sc] ^ sg] ^ sk]; \ - ctx->s[3][i] = mds[3][q1[q1[(a) ^ sd] ^ sh] ^ sl]; - -/* Macro exactly like CALC_SB_2, but for 256-bit keys. */ - -#define CALC_SB256_2(i, a, b) \ - ctx->s[0][i] = mds[0][q0[q0[q1[(b) ^ sa] ^ se] ^ si] ^ sm]; \ - ctx->s[1][i] = mds[1][q0[q1[q1[(a) ^ sb] ^ sf] ^ sj] ^ sn]; \ - ctx->s[2][i] = mds[2][q1[q0[q0[(a) ^ sc] ^ sg] ^ sk] ^ so]; \ - ctx->s[3][i] = mds[3][q1[q1[q0[(b) ^ sd] ^ sh] ^ sl] ^ sp]; - -/* Macros to calculate the whitening and round subkeys. CALC_K_2 computes the - * last two stages of the h() function for a given index (either 2i or 2i+1). - * a, b, c, and d are the four bytes going into the last two stages. For - * 128-bit keys, this is the entire h() function and a and c are the index - * preprocessed through q0 and q1 respectively; for longer keys they are the - * output of previous stages. j is the index of the first key byte to use. - * CALC_K computes a pair of subkeys for 128-bit Twofish, by calling CALC_K_2 - * twice, doing the Pseudo-Hadamard Transform, and doing the necessary - * rotations. Its parameters are: a, the array to write the results into, - * j, the index of the first output entry, k and l, the preprocessed indices - * for index 2i, and m and n, the preprocessed indices for index 2i+1. - * CALC_K192_2 expands CALC_K_2 to handle 192-bit keys, by doing an - * additional lookup-and-XOR stage. The parameters a, b, c and d are the - * four bytes going into the last three stages. For 192-bit keys, c = d - * are the index preprocessed through q0, and a = b are the index - * preprocessed through q1; j is the index of the first key byte to use. - * CALC_K192 is identical to CALC_K but for using the CALC_K192_2 macro - * instead of CALC_K_2. - * CALC_K256_2 expands CALC_K192_2 to handle 256-bit keys, by doing an - * additional lookup-and-XOR stage. The parameters a and b are the index - * preprocessed through q0 and q1 respectively; j is the index of the first - * key byte to use. CALC_K256 is identical to CALC_K but for using the - * CALC_K256_2 macro instead of CALC_K_2. */ - -#define CALC_K_2(a, b, c, d, j) \ - mds[0][q0[a ^ key[(j) + 8]] ^ key[j]] \ - ^ mds[1][q0[b ^ key[(j) + 9]] ^ key[(j) + 1]] \ - ^ mds[2][q1[c ^ key[(j) + 10]] ^ key[(j) + 2]] \ - ^ mds[3][q1[d ^ key[(j) + 11]] ^ key[(j) + 3]] - -#define CALC_K(a, j, k, l, m, n) \ - x = CALC_K_2 (k, l, k, l, 0); \ - y = CALC_K_2 (m, n, m, n, 4); \ - y = rol32(y, 8); \ - x += y; y += x; ctx->a[j] = x; \ - ctx->a[(j) + 1] = rol32(y, 9) - -#define CALC_K192_2(a, b, c, d, j) \ - CALC_K_2 (q0[a ^ key[(j) + 16]], \ - q1[b ^ key[(j) + 17]], \ - q0[c ^ key[(j) + 18]], \ - q1[d ^ key[(j) + 19]], j) - -#define CALC_K192(a, j, k, l, m, n) \ - x = CALC_K192_2 (l, l, k, k, 0); \ - y = CALC_K192_2 (n, n, m, m, 4); \ - y = rol32(y, 8); \ - x += y; y += x; ctx->a[j] = x; \ - ctx->a[(j) + 1] = rol32(y, 9) - -#define CALC_K256_2(a, b, j) \ - CALC_K192_2 (q1[b ^ key[(j) + 24]], \ - q1[a ^ key[(j) + 25]], \ - q0[a ^ key[(j) + 26]], \ - q0[b ^ key[(j) + 27]], j) - -#define CALC_K256(a, j, k, l, m, n) \ - x = CALC_K256_2 (k, l, 0); \ - y = CALC_K256_2 (m, n, 4); \ - y = rol32(y, 8); \ - x += y; y += x; ctx->a[j] = x; \ - ctx->a[(j) + 1] = rol32(y, 9) - - /* Macros to compute the g() function in the encryption and decryption * rounds. G1 is the straight g() function; G2 includes the 8-bit * rotation for the high 32-bit word. */ @@ -630,176 +103,7 @@ #define OUTUNPACK(n, x, m) \ x ^= ctx->w[m]; \ dst[n] = cpu_to_le32(x) -#define TF_MIN_KEY_SIZE 16 -#define TF_MAX_KEY_SIZE 32 -#define TF_BLOCK_SIZE 16 - -/* Structure for an expanded Twofish key. s contains the key-dependent - * S-boxes composed with the MDS matrix; w contains the eight "whitening" - * subkeys, K[0] through K[7]. k holds the remaining, "round" subkeys. Note - * that k[i] corresponds to what the Twofish paper calls K[i+8]. */ -struct twofish_ctx { - u32 s[4][256], w[8], k[32]; -}; - -/* Perform the key setup. */ -static int twofish_setkey(struct crypto_tfm *tfm, const u8 *key, - unsigned int key_len, u32 *flags) -{ - - struct twofish_ctx *ctx = crypto_tfm_ctx(tfm); - int i, j, k; - - /* Temporaries for CALC_K. */ - u32 x, y; - - /* The S vector used to key the S-boxes, split up into individual bytes. - * 128-bit keys use only sa through sh; 256-bit use all of them. */ - u8 sa = 0, sb = 0, sc = 0, sd = 0, se = 0, sf = 0, sg = 0, sh = 0; - u8 si = 0, sj = 0, sk = 0, sl = 0, sm = 0, sn = 0, so = 0, sp = 0; - - /* Temporary for CALC_S. */ - u8 tmp; - - /* Check key length. */ - if (key_len != 16 && key_len != 24 && key_len != 32) - { - *flags |= CRYPTO_TFM_RES_BAD_KEY_LEN; - return -EINVAL; /* unsupported key length */ - } - - /* Compute the first two words of the S vector. The magic numbers are - * the entries of the RS matrix, preprocessed through poly_to_exp. The - * numbers in the comments are the original (polynomial form) matrix - * entries. */ - CALC_S (sa, sb, sc, sd, 0, 0x00, 0x2D, 0x01, 0x2D); /* 01 A4 02 A4 */ - CALC_S (sa, sb, sc, sd, 1, 0x2D, 0xA4, 0x44, 0x8A); /* A4 56 A1 55 */ - CALC_S (sa, sb, sc, sd, 2, 0x8A, 0xD5, 0xBF, 0xD1); /* 55 82 FC 87 */ - CALC_S (sa, sb, sc, sd, 3, 0xD1, 0x7F, 0x3D, 0x99); /* 87 F3 C1 5A */ - CALC_S (sa, sb, sc, sd, 4, 0x99, 0x46, 0x66, 0x96); /* 5A 1E 47 58 */ - CALC_S (sa, sb, sc, sd, 5, 0x96, 0x3C, 0x5B, 0xED); /* 58 C6 AE DB */ - CALC_S (sa, sb, sc, sd, 6, 0xED, 0x37, 0x4F, 0xE0); /* DB 68 3D 9E */ - CALC_S (sa, sb, sc, sd, 7, 0xE0, 0xD0, 0x8C, 0x17); /* 9E E5 19 03 */ - CALC_S (se, sf, sg, sh, 8, 0x00, 0x2D, 0x01, 0x2D); /* 01 A4 02 A4 */ - CALC_S (se, sf, sg, sh, 9, 0x2D, 0xA4, 0x44, 0x8A); /* A4 56 A1 55 */ - CALC_S (se, sf, sg, sh, 10, 0x8A, 0xD5, 0xBF, 0xD1); /* 55 82 FC 87 */ - CALC_S (se, sf, sg, sh, 11, 0xD1, 0x7F, 0x3D, 0x99); /* 87 F3 C1 5A */ - CALC_S (se, sf, sg, sh, 12, 0x99, 0x46, 0x66, 0x96); /* 5A 1E 47 58 */ - CALC_S (se, sf, sg, sh, 13, 0x96, 0x3C, 0x5B, 0xED); /* 58 C6 AE DB */ - CALC_S (se, sf, sg, sh, 14, 0xED, 0x37, 0x4F, 0xE0); /* DB 68 3D 9E */ - CALC_S (se, sf, sg, sh, 15, 0xE0, 0xD0, 0x8C, 0x17); /* 9E E5 19 03 */ - - if (key_len == 24 || key_len == 32) { /* 192- or 256-bit key */ - /* Calculate the third word of the S vector */ - CALC_S (si, sj, sk, sl, 16, 0x00, 0x2D, 0x01, 0x2D); /* 01 A4 02 A4 */ - CALC_S (si, sj, sk, sl, 17, 0x2D, 0xA4, 0x44, 0x8A); /* A4 56 A1 55 */ - CALC_S (si, sj, sk, sl, 18, 0x8A, 0xD5, 0xBF, 0xD1); /* 55 82 FC 87 */ - CALC_S (si, sj, sk, sl, 19, 0xD1, 0x7F, 0x3D, 0x99); /* 87 F3 C1 5A */ - CALC_S (si, sj, sk, sl, 20, 0x99, 0x46, 0x66, 0x96); /* 5A 1E 47 58 */ - CALC_S (si, sj, sk, sl, 21, 0x96, 0x3C, 0x5B, 0xED); /* 58 C6 AE DB */ - CALC_S (si, sj, sk, sl, 22, 0xED, 0x37, 0x4F, 0xE0); /* DB 68 3D 9E */ - CALC_S (si, sj, sk, sl, 23, 0xE0, 0xD0, 0x8C, 0x17); /* 9E E5 19 03 */ - } - - if (key_len == 32) { /* 256-bit key */ - /* Calculate the fourth word of the S vector */ - CALC_S (sm, sn, so, sp, 24, 0x00, 0x2D, 0x01, 0x2D); /* 01 A4 02 A4 */ - CALC_S (sm, sn, so, sp, 25, 0x2D, 0xA4, 0x44, 0x8A); /* A4 56 A1 55 */ - CALC_S (sm, sn, so, sp, 26, 0x8A, 0xD5, 0xBF, 0xD1); /* 55 82 FC 87 */ - CALC_S (sm, sn, so, sp, 27, 0xD1, 0x7F, 0x3D, 0x99); /* 87 F3 C1 5A */ - CALC_S (sm, sn, so, sp, 28, 0x99, 0x46, 0x66, 0x96); /* 5A 1E 47 58 */ - CALC_S (sm, sn, so, sp, 29, 0x96, 0x3C, 0x5B, 0xED); /* 58 C6 AE DB */ - CALC_S (sm, sn, so, sp, 30, 0xED, 0x37, 0x4F, 0xE0); /* DB 68 3D 9E */ - CALC_S (sm, sn, so, sp, 31, 0xE0, 0xD0, 0x8C, 0x17); /* 9E E5 19 03 */ - - /* Compute the S-boxes. */ - for ( i = j = 0, k = 1; i < 256; i++, j += 2, k += 2 ) { - CALC_SB256_2( i, calc_sb_tbl[j], calc_sb_tbl[k] ); - } - - /* Calculate whitening and round subkeys. The constants are - * indices of subkeys, preprocessed through q0 and q1. */ - CALC_K256 (w, 0, 0xA9, 0x75, 0x67, 0xF3); - CALC_K256 (w, 2, 0xB3, 0xC6, 0xE8, 0xF4); - CALC_K256 (w, 4, 0x04, 0xDB, 0xFD, 0x7B); - CALC_K256 (w, 6, 0xA3, 0xFB, 0x76, 0xC8); - CALC_K256 (k, 0, 0x9A, 0x4A, 0x92, 0xD3); - CALC_K256 (k, 2, 0x80, 0xE6, 0x78, 0x6B); - CALC_K256 (k, 4, 0xE4, 0x45, 0xDD, 0x7D); - CALC_K256 (k, 6, 0xD1, 0xE8, 0x38, 0x4B); - CALC_K256 (k, 8, 0x0D, 0xD6, 0xC6, 0x32); - CALC_K256 (k, 10, 0x35, 0xD8, 0x98, 0xFD); - CALC_K256 (k, 12, 0x18, 0x37, 0xF7, 0x71); - CALC_K256 (k, 14, 0xEC, 0xF1, 0x6C, 0xE1); - CALC_K256 (k, 16, 0x43, 0x30, 0x75, 0x0F); - CALC_K256 (k, 18, 0x37, 0xF8, 0x26, 0x1B); - CALC_K256 (k, 20, 0xFA, 0x87, 0x13, 0xFA); - CALC_K256 (k, 22, 0x94, 0x06, 0x48, 0x3F); - CALC_K256 (k, 24, 0xF2, 0x5E, 0xD0, 0xBA); - CALC_K256 (k, 26, 0x8B, 0xAE, 0x30, 0x5B); - CALC_K256 (k, 28, 0x84, 0x8A, 0x54, 0x00); - CALC_K256 (k, 30, 0xDF, 0xBC, 0x23, 0x9D); - } else if (key_len == 24) { /* 192-bit key */ - /* Compute the S-boxes. */ - for ( i = j = 0, k = 1; i < 256; i++, j += 2, k += 2 ) { - CALC_SB192_2( i, calc_sb_tbl[j], calc_sb_tbl[k] ); - } - - /* Calculate whitening and round subkeys. The constants are - * indices of subkeys, preprocessed through q0 and q1. */ - CALC_K192 (w, 0, 0xA9, 0x75, 0x67, 0xF3); - CALC_K192 (w, 2, 0xB3, 0xC6, 0xE8, 0xF4); - CALC_K192 (w, 4, 0x04, 0xDB, 0xFD, 0x7B); - CALC_K192 (w, 6, 0xA3, 0xFB, 0x76, 0xC8); - CALC_K192 (k, 0, 0x9A, 0x4A, 0x92, 0xD3); - CALC_K192 (k, 2, 0x80, 0xE6, 0x78, 0x6B); - CALC_K192 (k, 4, 0xE4, 0x45, 0xDD, 0x7D); - CALC_K192 (k, 6, 0xD1, 0xE8, 0x38, 0x4B); - CALC_K192 (k, 8, 0x0D, 0xD6, 0xC6, 0x32); - CALC_K192 (k, 10, 0x35, 0xD8, 0x98, 0xFD); - CALC_K192 (k, 12, 0x18, 0x37, 0xF7, 0x71); - CALC_K192 (k, 14, 0xEC, 0xF1, 0x6C, 0xE1); - CALC_K192 (k, 16, 0x43, 0x30, 0x75, 0x0F); - CALC_K192 (k, 18, 0x37, 0xF8, 0x26, 0x1B); - CALC_K192 (k, 20, 0xFA, 0x87, 0x13, 0xFA); - CALC_K192 (k, 22, 0x94, 0x06, 0x48, 0x3F); - CALC_K192 (k, 24, 0xF2, 0x5E, 0xD0, 0xBA); - CALC_K192 (k, 26, 0x8B, 0xAE, 0x30, 0x5B); - CALC_K192 (k, 28, 0x84, 0x8A, 0x54, 0x00); - CALC_K192 (k, 30, 0xDF, 0xBC, 0x23, 0x9D); - } else { /* 128-bit key */ - /* Compute the S-boxes. */ - for ( i = j = 0, k = 1; i < 256; i++, j += 2, k += 2 ) { - CALC_SB_2( i, calc_sb_tbl[j], calc_sb_tbl[k] ); - } - - /* Calculate whitening and round subkeys. The constants are - * indices of subkeys, preprocessed through q0 and q1. */ - CALC_K (w, 0, 0xA9, 0x75, 0x67, 0xF3); - CALC_K (w, 2, 0xB3, 0xC6, 0xE8, 0xF4); - CALC_K (w, 4, 0x04, 0xDB, 0xFD, 0x7B); - CALC_K (w, 6, 0xA3, 0xFB, 0x76, 0xC8); - CALC_K (k, 0, 0x9A, 0x4A, 0x92, 0xD3); - CALC_K (k, 2, 0x80, 0xE6, 0x78, 0x6B); - CALC_K (k, 4, 0xE4, 0x45, 0xDD, 0x7D); - CALC_K (k, 6, 0xD1, 0xE8, 0x38, 0x4B); - CALC_K (k, 8, 0x0D, 0xD6, 0xC6, 0x32); - CALC_K (k, 10, 0x35, 0xD8, 0x98, 0xFD); - CALC_K (k, 12, 0x18, 0x37, 0xF7, 0x71); - CALC_K (k, 14, 0xEC, 0xF1, 0x6C, 0xE1); - CALC_K (k, 16, 0x43, 0x30, 0x75, 0x0F); - CALC_K (k, 18, 0x37, 0xF8, 0x26, 0x1B); - CALC_K (k, 20, 0xFA, 0x87, 0x13, 0xFA); - CALC_K (k, 22, 0x94, 0x06, 0x48, 0x3F); - CALC_K (k, 24, 0xF2, 0x5E, 0xD0, 0xBA); - CALC_K (k, 26, 0x8B, 0xAE, 0x30, 0x5B); - CALC_K (k, 28, 0x84, 0x8A, 0x54, 0x00); - CALC_K (k, 30, 0xDF, 0xBC, 0x23, 0x9D); - } - - return 0; -} /* Encrypt one block. in and out may be the same. */ static void twofish_encrypt(struct crypto_tfm *tfm, u8 *out, const u8 *in) @@ -877,6 +181,8 @@ static void twofish_decrypt(struct crypt static struct crypto_alg alg = { .cra_name = "twofish", + .cra_driver_name = "twofish-generic", + .cra_priority = 100, .cra_flags = CRYPTO_ALG_TYPE_CIPHER, .cra_blocksize = TF_BLOCK_SIZE, .cra_ctxsize = sizeof(struct twofish_ctx), diff --git a/crypto/twofish_common.c b/crypto/twofish_common.c new file mode 100644 index 0000000..b4b9c0c --- /dev/null +++ b/crypto/twofish_common.c @@ -0,0 +1,744 @@ +/* + * Common Twofish algorithm parts shared between the c and assembler + * implementations + * + * Originally Twofish for GPG + * By Matthew Skala , July 26, 1998 + * 256-bit key length added March 20, 1999 + * Some modifications to reduce the text size by Werner Koch, April, 1998 + * Ported to the kerneli patch by Marc Mutz + * Ported to CryptoAPI by Colin Slater + * + * The original author has disclaimed all copyright interest in this + * code and thus put it in the public domain. The subsequent authors + * have put this under the GNU General Public License. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 + * USA + * + * This code is a "clean room" implementation, written from the paper + * _Twofish: A 128-Bit Block Cipher_ by Bruce Schneier, John Kelsey, + * Doug Whiting, David Wagner, Chris Hall, and Niels Ferguson, available + * through http://www.counterpane.com/twofish.html + * + * For background information on multiplication in finite fields, used for + * the matrix operations in the key schedule, see the book _Contemporary + * Abstract Algebra_ by Joseph A. Gallian, especially chapter 22 in the + * Third Edition. + */ + +#include +#include +#include +#include +#include +#include +#include +#include + + +/* The large precomputed tables for the Twofish cipher (twofish.c) + * Taken from the same source as twofish.c + * Marc Mutz + */ + +/* These two tables are the q0 and q1 permutations, exactly as described in + * the Twofish paper. */ + +static const u8 q0[256] = { + 0xA9, 0x67, 0xB3, 0xE8, 0x04, 0xFD, 0xA3, 0x76, 0x9A, 0x92, 0x80, 0x78, + 0xE4, 0xDD, 0xD1, 0x38, 0x0D, 0xC6, 0x35, 0x98, 0x18, 0xF7, 0xEC, 0x6C, + 0x43, 0x75, 0x37, 0x26, 0xFA, 0x13, 0x94, 0x48, 0xF2, 0xD0, 0x8B, 0x30, + 0x84, 0x54, 0xDF, 0x23, 0x19, 0x5B, 0x3D, 0x59, 0xF3, 0xAE, 0xA2, 0x82, + 0x63, 0x01, 0x83, 0x2E, 0xD9, 0x51, 0x9B, 0x7C, 0xA6, 0xEB, 0xA5, 0xBE, + 0x16, 0x0C, 0xE3, 0x61, 0xC0, 0x8C, 0x3A, 0xF5, 0x73, 0x2C, 0x25, 0x0B, + 0xBB, 0x4E, 0x89, 0x6B, 0x53, 0x6A, 0xB4, 0xF1, 0xE1, 0xE6, 0xBD, 0x45, + 0xE2, 0xF4, 0xB6, 0x66, 0xCC, 0x95, 0x03, 0x56, 0xD4, 0x1C, 0x1E, 0xD7, + 0xFB, 0xC3, 0x8E, 0xB5, 0xE9, 0xCF, 0xBF, 0xBA, 0xEA, 0x77, 0x39, 0xAF, + 0x33, 0xC9, 0x62, 0x71, 0x81, 0x79, 0x09, 0xAD, 0x24, 0xCD, 0xF9, 0xD8, + 0xE5, 0xC5, 0xB9, 0x4D, 0x44, 0x08, 0x86, 0xE7, 0xA1, 0x1D, 0xAA, 0xED, + 0x06, 0x70, 0xB2, 0xD2, 0x41, 0x7B, 0xA0, 0x11, 0x31, 0xC2, 0x27, 0x90, + 0x20, 0xF6, 0x60, 0xFF, 0x96, 0x5C, 0xB1, 0xAB, 0x9E, 0x9C, 0x52, 0x1B, + 0x5F, 0x93, 0x0A, 0xEF, 0x91, 0x85, 0x49, 0xEE, 0x2D, 0x4F, 0x8F, 0x3B, + 0x47, 0x87, 0x6D, 0x46, 0xD6, 0x3E, 0x69, 0x64, 0x2A, 0xCE, 0xCB, 0x2F, + 0xFC, 0x97, 0x05, 0x7A, 0xAC, 0x7F, 0xD5, 0x1A, 0x4B, 0x0E, 0xA7, 0x5A, + 0x28, 0x14, 0x3F, 0x29, 0x88, 0x3C, 0x4C, 0x02, 0xB8, 0xDA, 0xB0, 0x17, + 0x55, 0x1F, 0x8A, 0x7D, 0x57, 0xC7, 0x8D, 0x74, 0xB7, 0xC4, 0x9F, 0x72, + 0x7E, 0x15, 0x22, 0x12, 0x58, 0x07, 0x99, 0x34, 0x6E, 0x50, 0xDE, 0x68, + 0x65, 0xBC, 0xDB, 0xF8, 0xC8, 0xA8, 0x2B, 0x40, 0xDC, 0xFE, 0x32, 0xA4, + 0xCA, 0x10, 0x21, 0xF0, 0xD3, 0x5D, 0x0F, 0x00, 0x6F, 0x9D, 0x36, 0x42, + 0x4A, 0x5E, 0xC1, 0xE0 +}; + +static const u8 q1[256] = { + 0x75, 0xF3, 0xC6, 0xF4, 0xDB, 0x7B, 0xFB, 0xC8, 0x4A, 0xD3, 0xE6, 0x6B, + 0x45, 0x7D, 0xE8, 0x4B, 0xD6, 0x32, 0xD8, 0xFD, 0x37, 0x71, 0xF1, 0xE1, + 0x30, 0x0F, 0xF8, 0x1B, 0x87, 0xFA, 0x06, 0x3F, 0x5E, 0xBA, 0xAE, 0x5B, + 0x8A, 0x00, 0xBC, 0x9D, 0x6D, 0xC1, 0xB1, 0x0E, 0x80, 0x5D, 0xD2, 0xD5, + 0xA0, 0x84, 0x07, 0x14, 0xB5, 0x90, 0x2C, 0xA3, 0xB2, 0x73, 0x4C, 0x54, + 0x92, 0x74, 0x36, 0x51, 0x38, 0xB0, 0xBD, 0x5A, 0xFC, 0x60, 0x62, 0x96, + 0x6C, 0x42, 0xF7, 0x10, 0x7C, 0x28, 0x27, 0x8C, 0x13, 0x95, 0x9C, 0xC7, + 0x24, 0x46, 0x3B, 0x70, 0xCA, 0xE3, 0x85, 0xCB, 0x11, 0xD0, 0x93, 0xB8, + 0xA6, 0x83, 0x20, 0xFF, 0x9F, 0x77, 0xC3, 0xCC, 0x03, 0x6F, 0x08, 0xBF, + 0x40, 0xE7, 0x2B, 0xE2, 0x79, 0x0C, 0xAA, 0x82, 0x41, 0x3A, 0xEA, 0xB9, + 0xE4, 0x9A, 0xA4, 0x97, 0x7E, 0xDA, 0x7A, 0x17, 0x66, 0x94, 0xA1, 0x1D, + 0x3D, 0xF0, 0xDE, 0xB3, 0x0B, 0x72, 0xA7, 0x1C, 0xEF, 0xD1, 0x53, 0x3E, + 0x8F, 0x33, 0x26, 0x5F, 0xEC, 0x76, 0x2A, 0x49, 0x81, 0x88, 0xEE, 0x21, + 0xC4, 0x1A, 0xEB, 0xD9, 0xC5, 0x39, 0x99, 0xCD, 0xAD, 0x31, 0x8B, 0x01, + 0x18, 0x23, 0xDD, 0x1F, 0x4E, 0x2D, 0xF9, 0x48, 0x4F, 0xF2, 0x65, 0x8E, + 0x78, 0x5C, 0x58, 0x19, 0x8D, 0xE5, 0x98, 0x57, 0x67, 0x7F, 0x05, 0x64, + 0xAF, 0x63, 0xB6, 0xFE, 0xF5, 0xB7, 0x3C, 0xA5, 0xCE, 0xE9, 0x68, 0x44, + 0xE0, 0x4D, 0x43, 0x69, 0x29, 0x2E, 0xAC, 0x15, 0x59, 0xA8, 0x0A, 0x9E, + 0x6E, 0x47, 0xDF, 0x34, 0x35, 0x6A, 0xCF, 0xDC, 0x22, 0xC9, 0xC0, 0x9B, + 0x89, 0xD4, 0xED, 0xAB, 0x12, 0xA2, 0x0D, 0x52, 0xBB, 0x02, 0x2F, 0xA9, + 0xD7, 0x61, 0x1E, 0xB4, 0x50, 0x04, 0xF6, 0xC2, 0x16, 0x25, 0x86, 0x56, + 0x55, 0x09, 0xBE, 0x91 +}; + +/* These MDS tables are actually tables of MDS composed with q0 and q1, + * because it is only ever used that way and we can save some time by + * precomputing. Of course the main saving comes from precomputing the + * GF(2^8) multiplication involved in the MDS matrix multiply; by looking + * things up in these tables we reduce the matrix multiply to four lookups + * and three XORs. Semi-formally, the definition of these tables is: + * mds[0][i] = MDS (q1[i] 0 0 0)^T mds[1][i] = MDS (0 q0[i] 0 0)^T + * mds[2][i] = MDS (0 0 q1[i] 0)^T mds[3][i] = MDS (0 0 0 q0[i])^T + * where ^T means "transpose", the matrix multiply is performed in GF(2^8) + * represented as GF(2)[x]/v(x) where v(x)=x^8+x^6+x^5+x^3+1 as described + * by Schneier et al, and I'm casually glossing over the byte/word + * conversion issues. */ + +static const u32 mds[4][256] = { + { + 0xBCBC3275, 0xECEC21F3, 0x202043C6, 0xB3B3C9F4, 0xDADA03DB, 0x02028B7B, + 0xE2E22BFB, 0x9E9EFAC8, 0xC9C9EC4A, 0xD4D409D3, 0x18186BE6, 0x1E1E9F6B, + 0x98980E45, 0xB2B2387D, 0xA6A6D2E8, 0x2626B74B, 0x3C3C57D6, 0x93938A32, + 0x8282EED8, 0x525298FD, 0x7B7BD437, 0xBBBB3771, 0x5B5B97F1, 0x474783E1, + 0x24243C30, 0x5151E20F, 0xBABAC6F8, 0x4A4AF31B, 0xBFBF4887, 0x0D0D70FA, + 0xB0B0B306, 0x7575DE3F, 0xD2D2FD5E, 0x7D7D20BA, 0x666631AE, 0x3A3AA35B, + 0x59591C8A, 0x00000000, 0xCDCD93BC, 0x1A1AE09D, 0xAEAE2C6D, 0x7F7FABC1, + 0x2B2BC7B1, 0xBEBEB90E, 0xE0E0A080, 0x8A8A105D, 0x3B3B52D2, 0x6464BAD5, + 0xD8D888A0, 0xE7E7A584, 0x5F5FE807, 0x1B1B1114, 0x2C2CC2B5, 0xFCFCB490, + 0x3131272C, 0x808065A3, 0x73732AB2, 0x0C0C8173, 0x79795F4C, 0x6B6B4154, + 0x4B4B0292, 0x53536974, 0x94948F36, 0x83831F51, 0x2A2A3638, 0xC4C49CB0, + 0x2222C8BD, 0xD5D5F85A, 0xBDBDC3FC, 0x48487860, 0xFFFFCE62, 0x4C4C0796, + 0x4141776C, 0xC7C7E642, 0xEBEB24F7, 0x1C1C1410, 0x5D5D637C, 0x36362228, + 0x6767C027, 0xE9E9AF8C, 0x4444F913, 0x1414EA95, 0xF5F5BB9C, 0xCFCF18C7, + 0x3F3F2D24, 0xC0C0E346, 0x7272DB3B, 0x54546C70, 0x29294CCA, 0xF0F035E3, + 0x0808FE85, 0xC6C617CB, 0xF3F34F11, 0x8C8CE4D0, 0xA4A45993, 0xCACA96B8, + 0x68683BA6, 0xB8B84D83, 0x38382820, 0xE5E52EFF, 0xADAD569F, 0x0B0B8477, + 0xC8C81DC3, 0x9999FFCC, 0x5858ED03, 0x19199A6F, 0x0E0E0A08, 0x95957EBF, + 0x70705040, 0xF7F730E7, 0x6E6ECF2B, 0x1F1F6EE2, 0xB5B53D79, 0x09090F0C, + 0x616134AA, 0x57571682, 0x9F9F0B41, 0x9D9D803A, 0x111164EA, 0x2525CDB9, + 0xAFAFDDE4, 0x4545089A, 0xDFDF8DA4, 0xA3A35C97, 0xEAEAD57E, 0x353558DA, + 0xEDEDD07A, 0x4343FC17, 0xF8F8CB66, 0xFBFBB194, 0x3737D3A1, 0xFAFA401D, + 0xC2C2683D, 0xB4B4CCF0, 0x32325DDE, 0x9C9C71B3, 0x5656E70B, 0xE3E3DA72, + 0x878760A7, 0x15151B1C, 0xF9F93AEF, 0x6363BFD1, 0x3434A953, 0x9A9A853E, + 0xB1B1428F, 0x7C7CD133, 0x88889B26, 0x3D3DA65F, 0xA1A1D7EC, 0xE4E4DF76, + 0x8181942A, 0x91910149, 0x0F0FFB81, 0xEEEEAA88, 0x161661EE, 0xD7D77321, + 0x9797F5C4, 0xA5A5A81A, 0xFEFE3FEB, 0x6D6DB5D9, 0x7878AEC5, 0xC5C56D39, + 0x1D1DE599, 0x7676A4CD, 0x3E3EDCAD, 0xCBCB6731, 0xB6B6478B, 0xEFEF5B01, + 0x12121E18, 0x6060C523, 0x6A6AB0DD, 0x4D4DF61F, 0xCECEE94E, 0xDEDE7C2D, + 0x55559DF9, 0x7E7E5A48, 0x2121B24F, 0x03037AF2, 0xA0A02665, 0x5E5E198E, + 0x5A5A6678, 0x65654B5C, 0x62624E58, 0xFDFD4519, 0x0606F48D, 0x404086E5, + 0xF2F2BE98, 0x3333AC57, 0x17179067, 0x05058E7F, 0xE8E85E05, 0x4F4F7D64, + 0x89896AAF, 0x10109563, 0x74742FB6, 0x0A0A75FE, 0x5C5C92F5, 0x9B9B74B7, + 0x2D2D333C, 0x3030D6A5, 0x2E2E49CE, 0x494989E9, 0x46467268, 0x77775544, + 0xA8A8D8E0, 0x9696044D, 0x2828BD43, 0xA9A92969, 0xD9D97929, 0x8686912E, + 0xD1D187AC, 0xF4F44A15, 0x8D8D1559, 0xD6D682A8, 0xB9B9BC0A, 0x42420D9E, + 0xF6F6C16E, 0x2F2FB847, 0xDDDD06DF, 0x23233934, 0xCCCC6235, 0xF1F1C46A, + 0xC1C112CF, 0x8585EBDC, 0x8F8F9E22, 0x7171A1C9, 0x9090F0C0, 0xAAAA539B, + 0x0101F189, 0x8B8BE1D4, 0x4E4E8CED, 0x8E8E6FAB, 0xABABA212, 0x6F6F3EA2, + 0xE6E6540D, 0xDBDBF252, 0x92927BBB, 0xB7B7B602, 0x6969CA2F, 0x3939D9A9, + 0xD3D30CD7, 0xA7A72361, 0xA2A2AD1E, 0xC3C399B4, 0x6C6C4450, 0x07070504, + 0x04047FF6, 0x272746C2, 0xACACA716, 0xD0D07625, 0x50501386, 0xDCDCF756, + 0x84841A55, 0xE1E15109, 0x7A7A25BE, 0x1313EF91}, + + { + 0xA9D93939, 0x67901717, 0xB3719C9C, 0xE8D2A6A6, 0x04050707, 0xFD985252, + 0xA3658080, 0x76DFE4E4, 0x9A084545, 0x92024B4B, 0x80A0E0E0, 0x78665A5A, + 0xE4DDAFAF, 0xDDB06A6A, 0xD1BF6363, 0x38362A2A, 0x0D54E6E6, 0xC6432020, + 0x3562CCCC, 0x98BEF2F2, 0x181E1212, 0xF724EBEB, 0xECD7A1A1, 0x6C774141, + 0x43BD2828, 0x7532BCBC, 0x37D47B7B, 0x269B8888, 0xFA700D0D, 0x13F94444, + 0x94B1FBFB, 0x485A7E7E, 0xF27A0303, 0xD0E48C8C, 0x8B47B6B6, 0x303C2424, + 0x84A5E7E7, 0x54416B6B, 0xDF06DDDD, 0x23C56060, 0x1945FDFD, 0x5BA33A3A, + 0x3D68C2C2, 0x59158D8D, 0xF321ECEC, 0xAE316666, 0xA23E6F6F, 0x82165757, + 0x63951010, 0x015BEFEF, 0x834DB8B8, 0x2E918686, 0xD9B56D6D, 0x511F8383, + 0x9B53AAAA, 0x7C635D5D, 0xA63B6868, 0xEB3FFEFE, 0xA5D63030, 0xBE257A7A, + 0x16A7ACAC, 0x0C0F0909, 0xE335F0F0, 0x6123A7A7, 0xC0F09090, 0x8CAFE9E9, + 0x3A809D9D, 0xF5925C5C, 0x73810C0C, 0x2C273131, 0x2576D0D0, 0x0BE75656, + 0xBB7B9292, 0x4EE9CECE, 0x89F10101, 0x6B9F1E1E, 0x53A93434, 0x6AC4F1F1, + 0xB499C3C3, 0xF1975B5B, 0xE1834747, 0xE66B1818, 0xBDC82222, 0x450E9898, + 0xE26E1F1F, 0xF4C9B3B3, 0xB62F7474, 0x66CBF8F8, 0xCCFF9999, 0x95EA1414, + 0x03ED5858, 0x56F7DCDC, 0xD4E18B8B, 0x1C1B1515, 0x1EADA2A2, 0xD70CD3D3, + 0xFB2BE2E2, 0xC31DC8C8, 0x8E195E5E, 0xB5C22C2C, 0xE9894949, 0xCF12C1C1, + 0xBF7E9595, 0xBA207D7D, 0xEA641111, 0x77840B0B, 0x396DC5C5, 0xAF6A8989, + 0x33D17C7C, 0xC9A17171, 0x62CEFFFF, 0x7137BBBB, 0x81FB0F0F, 0x793DB5B5, + 0x0951E1E1, 0xADDC3E3E, 0x242D3F3F, 0xCDA47676, 0xF99D5555, 0xD8EE8282, + 0xE5864040, 0xC5AE7878, 0xB9CD2525, 0x4D049696, 0x44557777, 0x080A0E0E, + 0x86135050, 0xE730F7F7, 0xA1D33737, 0x1D40FAFA, 0xAA346161, 0xED8C4E4E, + 0x06B3B0B0, 0x706C5454, 0xB22A7373, 0xD2523B3B, 0x410B9F9F, 0x7B8B0202, + 0xA088D8D8, 0x114FF3F3, 0x3167CBCB, 0xC2462727, 0x27C06767, 0x90B4FCFC, + 0x20283838, 0xF67F0404, 0x60784848, 0xFF2EE5E5, 0x96074C4C, 0x5C4B6565, + 0xB1C72B2B, 0xAB6F8E8E, 0x9E0D4242, 0x9CBBF5F5, 0x52F2DBDB, 0x1BF34A4A, + 0x5FA63D3D, 0x9359A4A4, 0x0ABCB9B9, 0xEF3AF9F9, 0x91EF1313, 0x85FE0808, + 0x49019191, 0xEE611616, 0x2D7CDEDE, 0x4FB22121, 0x8F42B1B1, 0x3BDB7272, + 0x47B82F2F, 0x8748BFBF, 0x6D2CAEAE, 0x46E3C0C0, 0xD6573C3C, 0x3E859A9A, + 0x6929A9A9, 0x647D4F4F, 0x2A948181, 0xCE492E2E, 0xCB17C6C6, 0x2FCA6969, + 0xFCC3BDBD, 0x975CA3A3, 0x055EE8E8, 0x7AD0EDED, 0xAC87D1D1, 0x7F8E0505, + 0xD5BA6464, 0x1AA8A5A5, 0x4BB72626, 0x0EB9BEBE, 0xA7608787, 0x5AF8D5D5, + 0x28223636, 0x14111B1B, 0x3FDE7575, 0x2979D9D9, 0x88AAEEEE, 0x3C332D2D, + 0x4C5F7979, 0x02B6B7B7, 0xB896CACA, 0xDA583535, 0xB09CC4C4, 0x17FC4343, + 0x551A8484, 0x1FF64D4D, 0x8A1C5959, 0x7D38B2B2, 0x57AC3333, 0xC718CFCF, + 0x8DF40606, 0x74695353, 0xB7749B9B, 0xC4F59797, 0x9F56ADAD, 0x72DAE3E3, + 0x7ED5EAEA, 0x154AF4F4, 0x229E8F8F, 0x12A2ABAB, 0x584E6262, 0x07E85F5F, + 0x99E51D1D, 0x34392323, 0x6EC1F6F6, 0x50446C6C, 0xDE5D3232, 0x68724646, + 0x6526A0A0, 0xBC93CDCD, 0xDB03DADA, 0xF8C6BABA, 0xC8FA9E9E, 0xA882D6D6, + 0x2BCF6E6E, 0x40507070, 0xDCEB8585, 0xFE750A0A, 0x328A9393, 0xA48DDFDF, + 0xCA4C2929, 0x10141C1C, 0x2173D7D7, 0xF0CCB4B4, 0xD309D4D4, 0x5D108A8A, + 0x0FE25151, 0x00000000, 0x6F9A1919, 0x9DE01A1A, 0x368F9494, 0x42E6C7C7, + 0x4AECC9C9, 0x5EFDD2D2, 0xC1AB7F7F, 0xE0D8A8A8}, + + { + 0xBC75BC32, 0xECF3EC21, 0x20C62043, 0xB3F4B3C9, 0xDADBDA03, 0x027B028B, + 0xE2FBE22B, 0x9EC89EFA, 0xC94AC9EC, 0xD4D3D409, 0x18E6186B, 0x1E6B1E9F, + 0x9845980E, 0xB27DB238, 0xA6E8A6D2, 0x264B26B7, 0x3CD63C57, 0x9332938A, + 0x82D882EE, 0x52FD5298, 0x7B377BD4, 0xBB71BB37, 0x5BF15B97, 0x47E14783, + 0x2430243C, 0x510F51E2, 0xBAF8BAC6, 0x4A1B4AF3, 0xBF87BF48, 0x0DFA0D70, + 0xB006B0B3, 0x753F75DE, 0xD25ED2FD, 0x7DBA7D20, 0x66AE6631, 0x3A5B3AA3, + 0x598A591C, 0x00000000, 0xCDBCCD93, 0x1A9D1AE0, 0xAE6DAE2C, 0x7FC17FAB, + 0x2BB12BC7, 0xBE0EBEB9, 0xE080E0A0, 0x8A5D8A10, 0x3BD23B52, 0x64D564BA, + 0xD8A0D888, 0xE784E7A5, 0x5F075FE8, 0x1B141B11, 0x2CB52CC2, 0xFC90FCB4, + 0x312C3127, 0x80A38065, 0x73B2732A, 0x0C730C81, 0x794C795F, 0x6B546B41, + 0x4B924B02, 0x53745369, 0x9436948F, 0x8351831F, 0x2A382A36, 0xC4B0C49C, + 0x22BD22C8, 0xD55AD5F8, 0xBDFCBDC3, 0x48604878, 0xFF62FFCE, 0x4C964C07, + 0x416C4177, 0xC742C7E6, 0xEBF7EB24, 0x1C101C14, 0x5D7C5D63, 0x36283622, + 0x672767C0, 0xE98CE9AF, 0x441344F9, 0x149514EA, 0xF59CF5BB, 0xCFC7CF18, + 0x3F243F2D, 0xC046C0E3, 0x723B72DB, 0x5470546C, 0x29CA294C, 0xF0E3F035, + 0x088508FE, 0xC6CBC617, 0xF311F34F, 0x8CD08CE4, 0xA493A459, 0xCAB8CA96, + 0x68A6683B, 0xB883B84D, 0x38203828, 0xE5FFE52E, 0xAD9FAD56, 0x0B770B84, + 0xC8C3C81D, 0x99CC99FF, 0x580358ED, 0x196F199A, 0x0E080E0A, 0x95BF957E, + 0x70407050, 0xF7E7F730, 0x6E2B6ECF, 0x1FE21F6E, 0xB579B53D, 0x090C090F, + 0x61AA6134, 0x57825716, 0x9F419F0B, 0x9D3A9D80, 0x11EA1164, 0x25B925CD, + 0xAFE4AFDD, 0x459A4508, 0xDFA4DF8D, 0xA397A35C, 0xEA7EEAD5, 0x35DA3558, + 0xED7AEDD0, 0x431743FC, 0xF866F8CB, 0xFB94FBB1, 0x37A137D3, 0xFA1DFA40, + 0xC23DC268, 0xB4F0B4CC, 0x32DE325D, 0x9CB39C71, 0x560B56E7, 0xE372E3DA, + 0x87A78760, 0x151C151B, 0xF9EFF93A, 0x63D163BF, 0x345334A9, 0x9A3E9A85, + 0xB18FB142, 0x7C337CD1, 0x8826889B, 0x3D5F3DA6, 0xA1ECA1D7, 0xE476E4DF, + 0x812A8194, 0x91499101, 0x0F810FFB, 0xEE88EEAA, 0x16EE1661, 0xD721D773, + 0x97C497F5, 0xA51AA5A8, 0xFEEBFE3F, 0x6DD96DB5, 0x78C578AE, 0xC539C56D, + 0x1D991DE5, 0x76CD76A4, 0x3EAD3EDC, 0xCB31CB67, 0xB68BB647, 0xEF01EF5B, + 0x1218121E, 0x602360C5, 0x6ADD6AB0, 0x4D1F4DF6, 0xCE4ECEE9, 0xDE2DDE7C, + 0x55F9559D, 0x7E487E5A, 0x214F21B2, 0x03F2037A, 0xA065A026, 0x5E8E5E19, + 0x5A785A66, 0x655C654B, 0x6258624E, 0xFD19FD45, 0x068D06F4, 0x40E54086, + 0xF298F2BE, 0x335733AC, 0x17671790, 0x057F058E, 0xE805E85E, 0x4F644F7D, + 0x89AF896A, 0x10631095, 0x74B6742F, 0x0AFE0A75, 0x5CF55C92, 0x9BB79B74, + 0x2D3C2D33, 0x30A530D6, 0x2ECE2E49, 0x49E94989, 0x46684672, 0x77447755, + 0xA8E0A8D8, 0x964D9604, 0x284328BD, 0xA969A929, 0xD929D979, 0x862E8691, + 0xD1ACD187, 0xF415F44A, 0x8D598D15, 0xD6A8D682, 0xB90AB9BC, 0x429E420D, + 0xF66EF6C1, 0x2F472FB8, 0xDDDFDD06, 0x23342339, 0xCC35CC62, 0xF16AF1C4, + 0xC1CFC112, 0x85DC85EB, 0x8F228F9E, 0x71C971A1, 0x90C090F0, 0xAA9BAA53, + 0x018901F1, 0x8BD48BE1, 0x4EED4E8C, 0x8EAB8E6F, 0xAB12ABA2, 0x6FA26F3E, + 0xE60DE654, 0xDB52DBF2, 0x92BB927B, 0xB702B7B6, 0x692F69CA, 0x39A939D9, + 0xD3D7D30C, 0xA761A723, 0xA21EA2AD, 0xC3B4C399, 0x6C506C44, 0x07040705, + 0x04F6047F, 0x27C22746, 0xAC16ACA7, 0xD025D076, 0x50865013, 0xDC56DCF7, + 0x8455841A, 0xE109E151, 0x7ABE7A25, 0x139113EF}, + + { + 0xD939A9D9, 0x90176790, 0x719CB371, 0xD2A6E8D2, 0x05070405, 0x9852FD98, + 0x6580A365, 0xDFE476DF, 0x08459A08, 0x024B9202, 0xA0E080A0, 0x665A7866, + 0xDDAFE4DD, 0xB06ADDB0, 0xBF63D1BF, 0x362A3836, 0x54E60D54, 0x4320C643, + 0x62CC3562, 0xBEF298BE, 0x1E12181E, 0x24EBF724, 0xD7A1ECD7, 0x77416C77, + 0xBD2843BD, 0x32BC7532, 0xD47B37D4, 0x9B88269B, 0x700DFA70, 0xF94413F9, + 0xB1FB94B1, 0x5A7E485A, 0x7A03F27A, 0xE48CD0E4, 0x47B68B47, 0x3C24303C, + 0xA5E784A5, 0x416B5441, 0x06DDDF06, 0xC56023C5, 0x45FD1945, 0xA33A5BA3, + 0x68C23D68, 0x158D5915, 0x21ECF321, 0x3166AE31, 0x3E6FA23E, 0x16578216, + 0x95106395, 0x5BEF015B, 0x4DB8834D, 0x91862E91, 0xB56DD9B5, 0x1F83511F, + 0x53AA9B53, 0x635D7C63, 0x3B68A63B, 0x3FFEEB3F, 0xD630A5D6, 0x257ABE25, + 0xA7AC16A7, 0x0F090C0F, 0x35F0E335, 0x23A76123, 0xF090C0F0, 0xAFE98CAF, + 0x809D3A80, 0x925CF592, 0x810C7381, 0x27312C27, 0x76D02576, 0xE7560BE7, + 0x7B92BB7B, 0xE9CE4EE9, 0xF10189F1, 0x9F1E6B9F, 0xA93453A9, 0xC4F16AC4, + 0x99C3B499, 0x975BF197, 0x8347E183, 0x6B18E66B, 0xC822BDC8, 0x0E98450E, + 0x6E1FE26E, 0xC9B3F4C9, 0x2F74B62F, 0xCBF866CB, 0xFF99CCFF, 0xEA1495EA, + 0xED5803ED, 0xF7DC56F7, 0xE18BD4E1, 0x1B151C1B, 0xADA21EAD, 0x0CD3D70C, + 0x2BE2FB2B, 0x1DC8C31D, 0x195E8E19, 0xC22CB5C2, 0x8949E989, 0x12C1CF12, + 0x7E95BF7E, 0x207DBA20, 0x6411EA64, 0x840B7784, 0x6DC5396D, 0x6A89AF6A, + 0xD17C33D1, 0xA171C9A1, 0xCEFF62CE, 0x37BB7137, 0xFB0F81FB, 0x3DB5793D, + 0x51E10951, 0xDC3EADDC, 0x2D3F242D, 0xA476CDA4, 0x9D55F99D, 0xEE82D8EE, + 0x8640E586, 0xAE78C5AE, 0xCD25B9CD, 0x04964D04, 0x55774455, 0x0A0E080A, + 0x13508613, 0x30F7E730, 0xD337A1D3, 0x40FA1D40, 0x3461AA34, 0x8C4EED8C, + 0xB3B006B3, 0x6C54706C, 0x2A73B22A, 0x523BD252, 0x0B9F410B, 0x8B027B8B, + 0x88D8A088, 0x4FF3114F, 0x67CB3167, 0x4627C246, 0xC06727C0, 0xB4FC90B4, + 0x28382028, 0x7F04F67F, 0x78486078, 0x2EE5FF2E, 0x074C9607, 0x4B655C4B, + 0xC72BB1C7, 0x6F8EAB6F, 0x0D429E0D, 0xBBF59CBB, 0xF2DB52F2, 0xF34A1BF3, + 0xA63D5FA6, 0x59A49359, 0xBCB90ABC, 0x3AF9EF3A, 0xEF1391EF, 0xFE0885FE, + 0x01914901, 0x6116EE61, 0x7CDE2D7C, 0xB2214FB2, 0x42B18F42, 0xDB723BDB, + 0xB82F47B8, 0x48BF8748, 0x2CAE6D2C, 0xE3C046E3, 0x573CD657, 0x859A3E85, + 0x29A96929, 0x7D4F647D, 0x94812A94, 0x492ECE49, 0x17C6CB17, 0xCA692FCA, + 0xC3BDFCC3, 0x5CA3975C, 0x5EE8055E, 0xD0ED7AD0, 0x87D1AC87, 0x8E057F8E, + 0xBA64D5BA, 0xA8A51AA8, 0xB7264BB7, 0xB9BE0EB9, 0x6087A760, 0xF8D55AF8, + 0x22362822, 0x111B1411, 0xDE753FDE, 0x79D92979, 0xAAEE88AA, 0x332D3C33, + 0x5F794C5F, 0xB6B702B6, 0x96CAB896, 0x5835DA58, 0x9CC4B09C, 0xFC4317FC, + 0x1A84551A, 0xF64D1FF6, 0x1C598A1C, 0x38B27D38, 0xAC3357AC, 0x18CFC718, + 0xF4068DF4, 0x69537469, 0x749BB774, 0xF597C4F5, 0x56AD9F56, 0xDAE372DA, + 0xD5EA7ED5, 0x4AF4154A, 0x9E8F229E, 0xA2AB12A2, 0x4E62584E, 0xE85F07E8, + 0xE51D99E5, 0x39233439, 0xC1F66EC1, 0x446C5044, 0x5D32DE5D, 0x72466872, + 0x26A06526, 0x93CDBC93, 0x03DADB03, 0xC6BAF8C6, 0xFA9EC8FA, 0x82D6A882, + 0xCF6E2BCF, 0x50704050, 0xEB85DCEB, 0x750AFE75, 0x8A93328A, 0x8DDFA48D, + 0x4C29CA4C, 0x141C1014, 0x73D72173, 0xCCB4F0CC, 0x09D4D309, 0x108A5D10, + 0xE2510FE2, 0x00000000, 0x9A196F9A, 0xE01A9DE0, 0x8F94368F, 0xE6C742E6, + 0xECC94AEC, 0xFDD25EFD, 0xAB7FC1AB, 0xD8A8E0D8} +}; + +/* The exp_to_poly and poly_to_exp tables are used to perform efficient + * operations in GF(2^8) represented as GF(2)[x]/w(x) where + * w(x)=x^8+x^6+x^3+x^2+1. We care about doing that because it's part of the + * definition of the RS matrix in the key schedule. Elements of that field + * are polynomials of degree not greater than 7 and all coefficients 0 or 1, + * which can be represented naturally by bytes (just substitute x=2). In that + * form, GF(2^8) addition is the same as bitwise XOR, but GF(2^8) + * multiplication is inefficient without hardware support. To multiply + * faster, I make use of the fact x is a generator for the nonzero elements, + * so that every element p of GF(2)[x]/w(x) is either 0 or equal to (x)^n for + * some n in 0..254. Note that that caret is exponentiation in GF(2^8), + * *not* polynomial notation. So if I want to compute pq where p and q are + * in GF(2^8), I can just say: + * 1. if p=0 or q=0 then pq=0 + * 2. otherwise, find m and n such that p=x^m and q=x^n + * 3. pq=(x^m)(x^n)=x^(m+n), so add m and n and find pq + * The translations in steps 2 and 3 are looked up in the tables + * poly_to_exp (for step 2) and exp_to_poly (for step 3). To see this + * in action, look at the CALC_S macro. As additional wrinkles, note that + * one of my operands is always a constant, so the poly_to_exp lookup on it + * is done in advance; I included the original values in the comments so + * readers can have some chance of recognizing that this *is* the RS matrix + * from the Twofish paper. I've only included the table entries I actually + * need; I never do a lookup on a variable input of zero and the biggest + * exponents I'll ever see are 254 (variable) and 237 (constant), so they'll + * never sum to more than 491. I'm repeating part of the exp_to_poly table + * so that I don't have to do mod-255 reduction in the exponent arithmetic. + * Since I know my constant operands are never zero, I only have to worry + * about zero values in the variable operand, and I do it with a simple + * conditional branch. I know conditionals are expensive, but I couldn't + * see a non-horrible way of avoiding them, and I did manage to group the + * statements so that each if covers four group multiplications. */ + +static const u8 poly_to_exp[255] = { + 0x00, 0x01, 0x17, 0x02, 0x2E, 0x18, 0x53, 0x03, 0x6A, 0x2F, 0x93, 0x19, + 0x34, 0x54, 0x45, 0x04, 0x5C, 0x6B, 0xB6, 0x30, 0xA6, 0x94, 0x4B, 0x1A, + 0x8C, 0x35, 0x81, 0x55, 0xAA, 0x46, 0x0D, 0x05, 0x24, 0x5D, 0x87, 0x6C, + 0x9B, 0xB7, 0xC1, 0x31, 0x2B, 0xA7, 0xA3, 0x95, 0x98, 0x4C, 0xCA, 0x1B, + 0xE6, 0x8D, 0x73, 0x36, 0xCD, 0x82, 0x12, 0x56, 0x62, 0xAB, 0xF0, 0x47, + 0x4F, 0x0E, 0xBD, 0x06, 0xD4, 0x25, 0xD2, 0x5E, 0x27, 0x88, 0x66, 0x6D, + 0xD6, 0x9C, 0x79, 0xB8, 0x08, 0xC2, 0xDF, 0x32, 0x68, 0x2C, 0xFD, 0xA8, + 0x8A, 0xA4, 0x5A, 0x96, 0x29, 0x99, 0x22, 0x4D, 0x60, 0xCB, 0xE4, 0x1C, + 0x7B, 0xE7, 0x3B, 0x8E, 0x9E, 0x74, 0xF4, 0x37, 0xD8, 0xCE, 0xF9, 0x83, + 0x6F, 0x13, 0xB2, 0x57, 0xE1, 0x63, 0xDC, 0xAC, 0xC4, 0xF1, 0xAF, 0x48, + 0x0A, 0x50, 0x42, 0x0F, 0xBA, 0xBE, 0xC7, 0x07, 0xDE, 0xD5, 0x78, 0x26, + 0x65, 0xD3, 0xD1, 0x5F, 0xE3, 0x28, 0x21, 0x89, 0x59, 0x67, 0xFC, 0x6E, + 0xB1, 0xD7, 0xF8, 0x9D, 0xF3, 0x7A, 0x3A, 0xB9, 0xC6, 0x09, 0x41, 0xC3, + 0xAE, 0xE0, 0xDB, 0x33, 0x44, 0x69, 0x92, 0x2D, 0x52, 0xFE, 0x16, 0xA9, + 0x0C, 0x8B, 0x80, 0xA5, 0x4A, 0x5B, 0xB5, 0x97, 0xC9, 0x2A, 0xA2, 0x9A, + 0xC0, 0x23, 0x86, 0x4E, 0xBC, 0x61, 0xEF, 0xCC, 0x11, 0xE5, 0x72, 0x1D, + 0x3D, 0x7C, 0xEB, 0xE8, 0xE9, 0x3C, 0xEA, 0x8F, 0x7D, 0x9F, 0xEC, 0x75, + 0x1E, 0xF5, 0x3E, 0x38, 0xF6, 0xD9, 0x3F, 0xCF, 0x76, 0xFA, 0x1F, 0x84, + 0xA0, 0x70, 0xED, 0x14, 0x90, 0xB3, 0x7E, 0x58, 0xFB, 0xE2, 0x20, 0x64, + 0xD0, 0xDD, 0x77, 0xAD, 0xDA, 0xC5, 0x40, 0xF2, 0x39, 0xB0, 0xF7, 0x49, + 0xB4, 0x0B, 0x7F, 0x51, 0x15, 0x43, 0x91, 0x10, 0x71, 0xBB, 0xEE, 0xBF, + 0x85, 0xC8, 0xA1 +}; + +static const u8 exp_to_poly[492] = { + 0x01, 0x02, 0x04, 0x08, 0x10, 0x20, 0x40, 0x80, 0x4D, 0x9A, 0x79, 0xF2, + 0xA9, 0x1F, 0x3E, 0x7C, 0xF8, 0xBD, 0x37, 0x6E, 0xDC, 0xF5, 0xA7, 0x03, + 0x06, 0x0C, 0x18, 0x30, 0x60, 0xC0, 0xCD, 0xD7, 0xE3, 0x8B, 0x5B, 0xB6, + 0x21, 0x42, 0x84, 0x45, 0x8A, 0x59, 0xB2, 0x29, 0x52, 0xA4, 0x05, 0x0A, + 0x14, 0x28, 0x50, 0xA0, 0x0D, 0x1A, 0x34, 0x68, 0xD0, 0xED, 0x97, 0x63, + 0xC6, 0xC1, 0xCF, 0xD3, 0xEB, 0x9B, 0x7B, 0xF6, 0xA1, 0x0F, 0x1E, 0x3C, + 0x78, 0xF0, 0xAD, 0x17, 0x2E, 0x5C, 0xB8, 0x3D, 0x7A, 0xF4, 0xA5, 0x07, + 0x0E, 0x1C, 0x38, 0x70, 0xE0, 0x8D, 0x57, 0xAE, 0x11, 0x22, 0x44, 0x88, + 0x5D, 0xBA, 0x39, 0x72, 0xE4, 0x85, 0x47, 0x8E, 0x51, 0xA2, 0x09, 0x12, + 0x24, 0x48, 0x90, 0x6D, 0xDA, 0xF9, 0xBF, 0x33, 0x66, 0xCC, 0xD5, 0xE7, + 0x83, 0x4B, 0x96, 0x61, 0xC2, 0xC9, 0xDF, 0xF3, 0xAB, 0x1B, 0x36, 0x6C, + 0xD8, 0xFD, 0xB7, 0x23, 0x46, 0x8C, 0x55, 0xAA, 0x19, 0x32, 0x64, 0xC8, + 0xDD, 0xF7, 0xA3, 0x0B, 0x16, 0x2C, 0x58, 0xB0, 0x2D, 0x5A, 0xB4, 0x25, + 0x4A, 0x94, 0x65, 0xCA, 0xD9, 0xFF, 0xB3, 0x2B, 0x56, 0xAC, 0x15, 0x2A, + 0x54, 0xA8, 0x1D, 0x3A, 0x74, 0xE8, 0x9D, 0x77, 0xEE, 0x91, 0x6F, 0xDE, + 0xF1, 0xAF, 0x13, 0x26, 0x4C, 0x98, 0x7D, 0xFA, 0xB9, 0x3F, 0x7E, 0xFC, + 0xB5, 0x27, 0x4E, 0x9C, 0x75, 0xEA, 0x99, 0x7F, 0xFE, 0xB1, 0x2F, 0x5E, + 0xBC, 0x35, 0x6A, 0xD4, 0xE5, 0x87, 0x43, 0x86, 0x41, 0x82, 0x49, 0x92, + 0x69, 0xD2, 0xE9, 0x9F, 0x73, 0xE6, 0x81, 0x4F, 0x9E, 0x71, 0xE2, 0x89, + 0x5F, 0xBE, 0x31, 0x62, 0xC4, 0xC5, 0xC7, 0xC3, 0xCB, 0xDB, 0xFB, 0xBB, + 0x3B, 0x76, 0xEC, 0x95, 0x67, 0xCE, 0xD1, 0xEF, 0x93, 0x6B, 0xD6, 0xE1, + 0x8F, 0x53, 0xA6, 0x01, 0x02, 0x04, 0x08, 0x10, 0x20, 0x40, 0x80, 0x4D, + 0x9A, 0x79, 0xF2, 0xA9, 0x1F, 0x3E, 0x7C, 0xF8, 0xBD, 0x37, 0x6E, 0xDC, + 0xF5, 0xA7, 0x03, 0x06, 0x0C, 0x18, 0x30, 0x60, 0xC0, 0xCD, 0xD7, 0xE3, + 0x8B, 0x5B, 0xB6, 0x21, 0x42, 0x84, 0x45, 0x8A, 0x59, 0xB2, 0x29, 0x52, + 0xA4, 0x05, 0x0A, 0x14, 0x28, 0x50, 0xA0, 0x0D, 0x1A, 0x34, 0x68, 0xD0, + 0xED, 0x97, 0x63, 0xC6, 0xC1, 0xCF, 0xD3, 0xEB, 0x9B, 0x7B, 0xF6, 0xA1, + 0x0F, 0x1E, 0x3C, 0x78, 0xF0, 0xAD, 0x17, 0x2E, 0x5C, 0xB8, 0x3D, 0x7A, + 0xF4, 0xA5, 0x07, 0x0E, 0x1C, 0x38, 0x70, 0xE0, 0x8D, 0x57, 0xAE, 0x11, + 0x22, 0x44, 0x88, 0x5D, 0xBA, 0x39, 0x72, 0xE4, 0x85, 0x47, 0x8E, 0x51, + 0xA2, 0x09, 0x12, 0x24, 0x48, 0x90, 0x6D, 0xDA, 0xF9, 0xBF, 0x33, 0x66, + 0xCC, 0xD5, 0xE7, 0x83, 0x4B, 0x96, 0x61, 0xC2, 0xC9, 0xDF, 0xF3, 0xAB, + 0x1B, 0x36, 0x6C, 0xD8, 0xFD, 0xB7, 0x23, 0x46, 0x8C, 0x55, 0xAA, 0x19, + 0x32, 0x64, 0xC8, 0xDD, 0xF7, 0xA3, 0x0B, 0x16, 0x2C, 0x58, 0xB0, 0x2D, + 0x5A, 0xB4, 0x25, 0x4A, 0x94, 0x65, 0xCA, 0xD9, 0xFF, 0xB3, 0x2B, 0x56, + 0xAC, 0x15, 0x2A, 0x54, 0xA8, 0x1D, 0x3A, 0x74, 0xE8, 0x9D, 0x77, 0xEE, + 0x91, 0x6F, 0xDE, 0xF1, 0xAF, 0x13, 0x26, 0x4C, 0x98, 0x7D, 0xFA, 0xB9, + 0x3F, 0x7E, 0xFC, 0xB5, 0x27, 0x4E, 0x9C, 0x75, 0xEA, 0x99, 0x7F, 0xFE, + 0xB1, 0x2F, 0x5E, 0xBC, 0x35, 0x6A, 0xD4, 0xE5, 0x87, 0x43, 0x86, 0x41, + 0x82, 0x49, 0x92, 0x69, 0xD2, 0xE9, 0x9F, 0x73, 0xE6, 0x81, 0x4F, 0x9E, + 0x71, 0xE2, 0x89, 0x5F, 0xBE, 0x31, 0x62, 0xC4, 0xC5, 0xC7, 0xC3, 0xCB +}; + + +/* The table constants are indices of + * S-box entries, preprocessed through q0 and q1. */ +static const u8 calc_sb_tbl[512] = { + 0xA9, 0x75, 0x67, 0xF3, 0xB3, 0xC6, 0xE8, 0xF4, + 0x04, 0xDB, 0xFD, 0x7B, 0xA3, 0xFB, 0x76, 0xC8, + 0x9A, 0x4A, 0x92, 0xD3, 0x80, 0xE6, 0x78, 0x6B, + 0xE4, 0x45, 0xDD, 0x7D, 0xD1, 0xE8, 0x38, 0x4B, + 0x0D, 0xD6, 0xC6, 0x32, 0x35, 0xD8, 0x98, 0xFD, + 0x18, 0x37, 0xF7, 0x71, 0xEC, 0xF1, 0x6C, 0xE1, + 0x43, 0x30, 0x75, 0x0F, 0x37, 0xF8, 0x26, 0x1B, + 0xFA, 0x87, 0x13, 0xFA, 0x94, 0x06, 0x48, 0x3F, + 0xF2, 0x5E, 0xD0, 0xBA, 0x8B, 0xAE, 0x30, 0x5B, + 0x84, 0x8A, 0x54, 0x00, 0xDF, 0xBC, 0x23, 0x9D, + 0x19, 0x6D, 0x5B, 0xC1, 0x3D, 0xB1, 0x59, 0x0E, + 0xF3, 0x80, 0xAE, 0x5D, 0xA2, 0xD2, 0x82, 0xD5, + 0x63, 0xA0, 0x01, 0x84, 0x83, 0x07, 0x2E, 0x14, + 0xD9, 0xB5, 0x51, 0x90, 0x9B, 0x2C, 0x7C, 0xA3, + 0xA6, 0xB2, 0xEB, 0x73, 0xA5, 0x4C, 0xBE, 0x54, + 0x16, 0x92, 0x0C, 0x74, 0xE3, 0x36, 0x61, 0x51, + 0xC0, 0x38, 0x8C, 0xB0, 0x3A, 0xBD, 0xF5, 0x5A, + 0x73, 0xFC, 0x2C, 0x60, 0x25, 0x62, 0x0B, 0x96, + 0xBB, 0x6C, 0x4E, 0x42, 0x89, 0xF7, 0x6B, 0x10, + 0x53, 0x7C, 0x6A, 0x28, 0xB4, 0x27, 0xF1, 0x8C, + 0xE1, 0x13, 0xE6, 0x95, 0xBD, 0x9C, 0x45, 0xC7, + 0xE2, 0x24, 0xF4, 0x46, 0xB6, 0x3B, 0x66, 0x70, + 0xCC, 0xCA, 0x95, 0xE3, 0x03, 0x85, 0x56, 0xCB, + 0xD4, 0x11, 0x1C, 0xD0, 0x1E, 0x93, 0xD7, 0xB8, + 0xFB, 0xA6, 0xC3, 0x83, 0x8E, 0x20, 0xB5, 0xFF, + 0xE9, 0x9F, 0xCF, 0x77, 0xBF, 0xC3, 0xBA, 0xCC, + 0xEA, 0x03, 0x77, 0x6F, 0x39, 0x08, 0xAF, 0xBF, + 0x33, 0x40, 0xC9, 0xE7, 0x62, 0x2B, 0x71, 0xE2, + 0x81, 0x79, 0x79, 0x0C, 0x09, 0xAA, 0xAD, 0x82, + 0x24, 0x41, 0xCD, 0x3A, 0xF9, 0xEA, 0xD8, 0xB9, + 0xE5, 0xE4, 0xC5, 0x9A, 0xB9, 0xA4, 0x4D, 0x97, + 0x44, 0x7E, 0x08, 0xDA, 0x86, 0x7A, 0xE7, 0x17, + 0xA1, 0x66, 0x1D, 0x94, 0xAA, 0xA1, 0xED, 0x1D, + 0x06, 0x3D, 0x70, 0xF0, 0xB2, 0xDE, 0xD2, 0xB3, + 0x41, 0x0B, 0x7B, 0x72, 0xA0, 0xA7, 0x11, 0x1C, + 0x31, 0xEF, 0xC2, 0xD1, 0x27, 0x53, 0x90, 0x3E, + 0x20, 0x8F, 0xF6, 0x33, 0x60, 0x26, 0xFF, 0x5F, + 0x96, 0xEC, 0x5C, 0x76, 0xB1, 0x2A, 0xAB, 0x49, + 0x9E, 0x81, 0x9C, 0x88, 0x52, 0xEE, 0x1B, 0x21, + 0x5F, 0xC4, 0x93, 0x1A, 0x0A, 0xEB, 0xEF, 0xD9, + 0x91, 0xC5, 0x85, 0x39, 0x49, 0x99, 0xEE, 0xCD, + 0x2D, 0xAD, 0x4F, 0x31, 0x8F, 0x8B, 0x3B, 0x01, + 0x47, 0x18, 0x87, 0x23, 0x6D, 0xDD, 0x46, 0x1F, + 0xD6, 0x4E, 0x3E, 0x2D, 0x69, 0xF9, 0x64, 0x48, + 0x2A, 0x4F, 0xCE, 0xF2, 0xCB, 0x65, 0x2F, 0x8E, + 0xFC, 0x78, 0x97, 0x5C, 0x05, 0x58, 0x7A, 0x19, + 0xAC, 0x8D, 0x7F, 0xE5, 0xD5, 0x98, 0x1A, 0x57, + 0x4B, 0x67, 0x0E, 0x7F, 0xA7, 0x05, 0x5A, 0x64, + 0x28, 0xAF, 0x14, 0x63, 0x3F, 0xB6, 0x29, 0xFE, + 0x88, 0xF5, 0x3C, 0xB7, 0x4C, 0x3C, 0x02, 0xA5, + 0xB8, 0xCE, 0xDA, 0xE9, 0xB0, 0x68, 0x17, 0x44, + 0x55, 0xE0, 0x1F, 0x4D, 0x8A, 0x43, 0x7D, 0x69, + 0x57, 0x29, 0xC7, 0x2E, 0x8D, 0xAC, 0x74, 0x15, + 0xB7, 0x59, 0xC4, 0xA8, 0x9F, 0x0A, 0x72, 0x9E, + 0x7E, 0x6E, 0x15, 0x47, 0x22, 0xDF, 0x12, 0x34, + 0x58, 0x35, 0x07, 0x6A, 0x99, 0xCF, 0x34, 0xDC, + 0x6E, 0x22, 0x50, 0xC9, 0xDE, 0xC0, 0x68, 0x9B, + 0x65, 0x89, 0xBC, 0xD4, 0xDB, 0xED, 0xF8, 0xAB, + 0xC8, 0x12, 0xA8, 0xA2, 0x2B, 0x0D, 0x40, 0x52, + 0xDC, 0xBB, 0xFE, 0x02, 0x32, 0x2F, 0xA4, 0xA9, + 0xCA, 0xD7, 0x10, 0x61, 0x21, 0x1E, 0xF0, 0xB4, + 0xD3, 0x50, 0x5D, 0x04, 0x0F, 0xF6, 0x00, 0xC2, + 0x6F, 0x16, 0x9D, 0x25, 0x36, 0x86, 0x42, 0x56, + 0x4A, 0x55, 0x5E, 0x09, 0xC1, 0xBE, 0xE0, 0x91 +}; + +/* Macro to perform one column of the RS matrix multiplication. The + * parameters a, b, c, and d are the four bytes of output; i is the index + * of the key bytes, and w, x, y, and z, are the column of constants from + * the RS matrix, preprocessed through the poly_to_exp table. */ + +#define CALC_S(a, b, c, d, i, w, x, y, z) \ + if (key[i]) { \ + tmp = poly_to_exp[key[i] - 1]; \ + (a) ^= exp_to_poly[tmp + (w)]; \ + (b) ^= exp_to_poly[tmp + (x)]; \ + (c) ^= exp_to_poly[tmp + (y)]; \ + (d) ^= exp_to_poly[tmp + (z)]; \ + } + +/* Macros to calculate the key-dependent S-boxes for a 128-bit key using + * the S vector from CALC_S. CALC_SB_2 computes a single entry in all + * four S-boxes, where i is the index of the entry to compute, and a and b + * are the index numbers preprocessed through the q0 and q1 tables + * respectively. */ + +#define CALC_SB_2(i, a, b) \ + ctx->s[0][i] = mds[0][q0[(a) ^ sa] ^ se]; \ + ctx->s[1][i] = mds[1][q0[(b) ^ sb] ^ sf]; \ + ctx->s[2][i] = mds[2][q1[(a) ^ sc] ^ sg]; \ + ctx->s[3][i] = mds[3][q1[(b) ^ sd] ^ sh] + +/* Macro exactly like CALC_SB_2, but for 192-bit keys. */ + +#define CALC_SB192_2(i, a, b) \ + ctx->s[0][i] = mds[0][q0[q0[(b) ^ sa] ^ se] ^ si]; \ + ctx->s[1][i] = mds[1][q0[q1[(b) ^ sb] ^ sf] ^ sj]; \ + ctx->s[2][i] = mds[2][q1[q0[(a) ^ sc] ^ sg] ^ sk]; \ + ctx->s[3][i] = mds[3][q1[q1[(a) ^ sd] ^ sh] ^ sl]; + +/* Macro exactly like CALC_SB_2, but for 256-bit keys. */ + +#define CALC_SB256_2(i, a, b) \ + ctx->s[0][i] = mds[0][q0[q0[q1[(b) ^ sa] ^ se] ^ si] ^ sm]; \ + ctx->s[1][i] = mds[1][q0[q1[q1[(a) ^ sb] ^ sf] ^ sj] ^ sn]; \ + ctx->s[2][i] = mds[2][q1[q0[q0[(a) ^ sc] ^ sg] ^ sk] ^ so]; \ + ctx->s[3][i] = mds[3][q1[q1[q0[(b) ^ sd] ^ sh] ^ sl] ^ sp]; + +/* Macros to calculate the whitening and round subkeys. CALC_K_2 computes the + * last two stages of the h() function for a given index (either 2i or 2i+1). + * a, b, c, and d are the four bytes going into the last two stages. For + * 128-bit keys, this is the entire h() function and a and c are the index + * preprocessed through q0 and q1 respectively; for longer keys they are the + * output of previous stages. j is the index of the first key byte to use. + * CALC_K computes a pair of subkeys for 128-bit Twofish, by calling CALC_K_2 + * twice, doing the Pseudo-Hadamard Transform, and doing the necessary + * rotations. Its parameters are: a, the array to write the results into, + * j, the index of the first output entry, k and l, the preprocessed indices + * for index 2i, and m and n, the preprocessed indices for index 2i+1. + * CALC_K192_2 expands CALC_K_2 to handle 192-bit keys, by doing an + * additional lookup-and-XOR stage. The parameters a, b, c and d are the + * four bytes going into the last three stages. For 192-bit keys, c = d + * are the index preprocessed through q0, and a = b are the index + * preprocessed through q1; j is the index of the first key byte to use. + * CALC_K192 is identical to CALC_K but for using the CALC_K192_2 macro + * instead of CALC_K_2. + * CALC_K256_2 expands CALC_K192_2 to handle 256-bit keys, by doing an + * additional lookup-and-XOR stage. The parameters a and b are the index + * preprocessed through q0 and q1 respectively; j is the index of the first + * key byte to use. CALC_K256 is identical to CALC_K but for using the + * CALC_K256_2 macro instead of CALC_K_2. */ + +#define CALC_K_2(a, b, c, d, j) \ + mds[0][q0[a ^ key[(j) + 8]] ^ key[j]] \ + ^ mds[1][q0[b ^ key[(j) + 9]] ^ key[(j) + 1]] \ + ^ mds[2][q1[c ^ key[(j) + 10]] ^ key[(j) + 2]] \ + ^ mds[3][q1[d ^ key[(j) + 11]] ^ key[(j) + 3]] + +#define CALC_K(a, j, k, l, m, n) \ + x = CALC_K_2 (k, l, k, l, 0); \ + y = CALC_K_2 (m, n, m, n, 4); \ + y = rol32(y, 8); \ + x += y; y += x; ctx->a[j] = x; \ + ctx->a[(j) + 1] = rol32(y, 9) + +#define CALC_K192_2(a, b, c, d, j) \ + CALC_K_2 (q0[a ^ key[(j) + 16]], \ + q1[b ^ key[(j) + 17]], \ + q0[c ^ key[(j) + 18]], \ + q1[d ^ key[(j) + 19]], j) + +#define CALC_K192(a, j, k, l, m, n) \ + x = CALC_K192_2 (l, l, k, k, 0); \ + y = CALC_K192_2 (n, n, m, m, 4); \ + y = rol32(y, 8); \ + x += y; y += x; ctx->a[j] = x; \ + ctx->a[(j) + 1] = rol32(y, 9) + +#define CALC_K256_2(a, b, j) \ + CALC_K192_2 (q1[b ^ key[(j) + 24]], \ + q1[a ^ key[(j) + 25]], \ + q0[a ^ key[(j) + 26]], \ + q0[b ^ key[(j) + 27]], j) + +#define CALC_K256(a, j, k, l, m, n) \ + x = CALC_K256_2 (k, l, 0); \ + y = CALC_K256_2 (m, n, 4); \ + y = rol32(y, 8); \ + x += y; y += x; ctx->a[j] = x; \ + ctx->a[(j) + 1] = rol32(y, 9) + +/* Perform the key setup. */ +int twofish_setkey(struct crypto_tfm *tfm, const u8 *key, unsigned int key_len) +{ + + struct twofish_ctx *ctx = crypto_tfm_ctx(tfm); + u32 *flags = &tfm->crt_flags; + + int i, j, k; + + /* Temporaries for CALC_K. */ + u32 x, y; + + /* The S vector used to key the S-boxes, split up into individual bytes. + * 128-bit keys use only sa through sh; 256-bit use all of them. */ + u8 sa = 0, sb = 0, sc = 0, sd = 0, se = 0, sf = 0, sg = 0, sh = 0; + u8 si = 0, sj = 0, sk = 0, sl = 0, sm = 0, sn = 0, so = 0, sp = 0; + + /* Temporary for CALC_S. */ + u8 tmp; + + /* Check key length. */ + if (key_len % 8) + { + *flags |= CRYPTO_TFM_RES_BAD_KEY_LEN; + return -EINVAL; /* unsupported key length */ + } + + /* Compute the first two words of the S vector. The magic numbers are + * the entries of the RS matrix, preprocessed through poly_to_exp. The + * numbers in the comments are the original (polynomial form) matrix + * entries. */ + CALC_S (sa, sb, sc, sd, 0, 0x00, 0x2D, 0x01, 0x2D); /* 01 A4 02 A4 */ + CALC_S (sa, sb, sc, sd, 1, 0x2D, 0xA4, 0x44, 0x8A); /* A4 56 A1 55 */ + CALC_S (sa, sb, sc, sd, 2, 0x8A, 0xD5, 0xBF, 0xD1); /* 55 82 FC 87 */ + CALC_S (sa, sb, sc, sd, 3, 0xD1, 0x7F, 0x3D, 0x99); /* 87 F3 C1 5A */ + CALC_S (sa, sb, sc, sd, 4, 0x99, 0x46, 0x66, 0x96); /* 5A 1E 47 58 */ + CALC_S (sa, sb, sc, sd, 5, 0x96, 0x3C, 0x5B, 0xED); /* 58 C6 AE DB */ + CALC_S (sa, sb, sc, sd, 6, 0xED, 0x37, 0x4F, 0xE0); /* DB 68 3D 9E */ + CALC_S (sa, sb, sc, sd, 7, 0xE0, 0xD0, 0x8C, 0x17); /* 9E E5 19 03 */ + CALC_S (se, sf, sg, sh, 8, 0x00, 0x2D, 0x01, 0x2D); /* 01 A4 02 A4 */ + CALC_S (se, sf, sg, sh, 9, 0x2D, 0xA4, 0x44, 0x8A); /* A4 56 A1 55 */ + CALC_S (se, sf, sg, sh, 10, 0x8A, 0xD5, 0xBF, 0xD1); /* 55 82 FC 87 */ + CALC_S (se, sf, sg, sh, 11, 0xD1, 0x7F, 0x3D, 0x99); /* 87 F3 C1 5A */ + CALC_S (se, sf, sg, sh, 12, 0x99, 0x46, 0x66, 0x96); /* 5A 1E 47 58 */ + CALC_S (se, sf, sg, sh, 13, 0x96, 0x3C, 0x5B, 0xED); /* 58 C6 AE DB */ + CALC_S (se, sf, sg, sh, 14, 0xED, 0x37, 0x4F, 0xE0); /* DB 68 3D 9E */ + CALC_S (se, sf, sg, sh, 15, 0xE0, 0xD0, 0x8C, 0x17); /* 9E E5 19 03 */ + + if (key_len == 24 || key_len == 32) { /* 192- or 256-bit key */ + /* Calculate the third word of the S vector */ + CALC_S (si, sj, sk, sl, 16, 0x00, 0x2D, 0x01, 0x2D); /* 01 A4 02 A4 */ + CALC_S (si, sj, sk, sl, 17, 0x2D, 0xA4, 0x44, 0x8A); /* A4 56 A1 55 */ + CALC_S (si, sj, sk, sl, 18, 0x8A, 0xD5, 0xBF, 0xD1); /* 55 82 FC 87 */ + CALC_S (si, sj, sk, sl, 19, 0xD1, 0x7F, 0x3D, 0x99); /* 87 F3 C1 5A */ + CALC_S (si, sj, sk, sl, 20, 0x99, 0x46, 0x66, 0x96); /* 5A 1E 47 58 */ + CALC_S (si, sj, sk, sl, 21, 0x96, 0x3C, 0x5B, 0xED); /* 58 C6 AE DB */ + CALC_S (si, sj, sk, sl, 22, 0xED, 0x37, 0x4F, 0xE0); /* DB 68 3D 9E */ + CALC_S (si, sj, sk, sl, 23, 0xE0, 0xD0, 0x8C, 0x17); /* 9E E5 19 03 */ + } + + if (key_len == 32) { /* 256-bit key */ + /* Calculate the fourth word of the S vector */ + CALC_S (sm, sn, so, sp, 24, 0x00, 0x2D, 0x01, 0x2D); /* 01 A4 02 A4 */ + CALC_S (sm, sn, so, sp, 25, 0x2D, 0xA4, 0x44, 0x8A); /* A4 56 A1 55 */ + CALC_S (sm, sn, so, sp, 26, 0x8A, 0xD5, 0xBF, 0xD1); /* 55 82 FC 87 */ + CALC_S (sm, sn, so, sp, 27, 0xD1, 0x7F, 0x3D, 0x99); /* 87 F3 C1 5A */ + CALC_S (sm, sn, so, sp, 28, 0x99, 0x46, 0x66, 0x96); /* 5A 1E 47 58 */ + CALC_S (sm, sn, so, sp, 29, 0x96, 0x3C, 0x5B, 0xED); /* 58 C6 AE DB */ + CALC_S (sm, sn, so, sp, 30, 0xED, 0x37, 0x4F, 0xE0); /* DB 68 3D 9E */ + CALC_S (sm, sn, so, sp, 31, 0xE0, 0xD0, 0x8C, 0x17); /* 9E E5 19 03 */ + + /* Compute the S-boxes. */ + for ( i = j = 0, k = 1; i < 256; i++, j += 2, k += 2 ) { + CALC_SB256_2( i, calc_sb_tbl[j], calc_sb_tbl[k] ); + } + + /* Calculate whitening and round subkeys. The constants are + * indices of subkeys, preprocessed through q0 and q1. */ + CALC_K256 (w, 0, 0xA9, 0x75, 0x67, 0xF3); + CALC_K256 (w, 2, 0xB3, 0xC6, 0xE8, 0xF4); + CALC_K256 (w, 4, 0x04, 0xDB, 0xFD, 0x7B); + CALC_K256 (w, 6, 0xA3, 0xFB, 0x76, 0xC8); + CALC_K256 (k, 0, 0x9A, 0x4A, 0x92, 0xD3); + CALC_K256 (k, 2, 0x80, 0xE6, 0x78, 0x6B); + CALC_K256 (k, 4, 0xE4, 0x45, 0xDD, 0x7D); + CALC_K256 (k, 6, 0xD1, 0xE8, 0x38, 0x4B); + CALC_K256 (k, 8, 0x0D, 0xD6, 0xC6, 0x32); + CALC_K256 (k, 10, 0x35, 0xD8, 0x98, 0xFD); + CALC_K256 (k, 12, 0x18, 0x37, 0xF7, 0x71); + CALC_K256 (k, 14, 0xEC, 0xF1, 0x6C, 0xE1); + CALC_K256 (k, 16, 0x43, 0x30, 0x75, 0x0F); + CALC_K256 (k, 18, 0x37, 0xF8, 0x26, 0x1B); + CALC_K256 (k, 20, 0xFA, 0x87, 0x13, 0xFA); + CALC_K256 (k, 22, 0x94, 0x06, 0x48, 0x3F); + CALC_K256 (k, 24, 0xF2, 0x5E, 0xD0, 0xBA); + CALC_K256 (k, 26, 0x8B, 0xAE, 0x30, 0x5B); + CALC_K256 (k, 28, 0x84, 0x8A, 0x54, 0x00); + CALC_K256 (k, 30, 0xDF, 0xBC, 0x23, 0x9D); + } else if (key_len == 24) { /* 192-bit key */ + /* Compute the S-boxes. */ + for ( i = j = 0, k = 1; i < 256; i++, j += 2, k += 2 ) { + CALC_SB192_2( i, calc_sb_tbl[j], calc_sb_tbl[k] ); + } + + /* Calculate whitening and round subkeys. The constants are + * indices of subkeys, preprocessed through q0 and q1. */ + CALC_K192 (w, 0, 0xA9, 0x75, 0x67, 0xF3); + CALC_K192 (w, 2, 0xB3, 0xC6, 0xE8, 0xF4); + CALC_K192 (w, 4, 0x04, 0xDB, 0xFD, 0x7B); + CALC_K192 (w, 6, 0xA3, 0xFB, 0x76, 0xC8); + CALC_K192 (k, 0, 0x9A, 0x4A, 0x92, 0xD3); + CALC_K192 (k, 2, 0x80, 0xE6, 0x78, 0x6B); + CALC_K192 (k, 4, 0xE4, 0x45, 0xDD, 0x7D); + CALC_K192 (k, 6, 0xD1, 0xE8, 0x38, 0x4B); + CALC_K192 (k, 8, 0x0D, 0xD6, 0xC6, 0x32); + CALC_K192 (k, 10, 0x35, 0xD8, 0x98, 0xFD); + CALC_K192 (k, 12, 0x18, 0x37, 0xF7, 0x71); + CALC_K192 (k, 14, 0xEC, 0xF1, 0x6C, 0xE1); + CALC_K192 (k, 16, 0x43, 0x30, 0x75, 0x0F); + CALC_K192 (k, 18, 0x37, 0xF8, 0x26, 0x1B); + CALC_K192 (k, 20, 0xFA, 0x87, 0x13, 0xFA); + CALC_K192 (k, 22, 0x94, 0x06, 0x48, 0x3F); + CALC_K192 (k, 24, 0xF2, 0x5E, 0xD0, 0xBA); + CALC_K192 (k, 26, 0x8B, 0xAE, 0x30, 0x5B); + CALC_K192 (k, 28, 0x84, 0x8A, 0x54, 0x00); + CALC_K192 (k, 30, 0xDF, 0xBC, 0x23, 0x9D); + } else { /* 128-bit key */ + /* Compute the S-boxes. */ + for ( i = j = 0, k = 1; i < 256; i++, j += 2, k += 2 ) { + CALC_SB_2( i, calc_sb_tbl[j], calc_sb_tbl[k] ); + } + + /* Calculate whitening and round subkeys. The constants are + * indices of subkeys, preprocessed through q0 and q1. */ + CALC_K (w, 0, 0xA9, 0x75, 0x67, 0xF3); + CALC_K (w, 2, 0xB3, 0xC6, 0xE8, 0xF4); + CALC_K (w, 4, 0x04, 0xDB, 0xFD, 0x7B); + CALC_K (w, 6, 0xA3, 0xFB, 0x76, 0xC8); + CALC_K (k, 0, 0x9A, 0x4A, 0x92, 0xD3); + CALC_K (k, 2, 0x80, 0xE6, 0x78, 0x6B); + CALC_K (k, 4, 0xE4, 0x45, 0xDD, 0x7D); + CALC_K (k, 6, 0xD1, 0xE8, 0x38, 0x4B); + CALC_K (k, 8, 0x0D, 0xD6, 0xC6, 0x32); + CALC_K (k, 10, 0x35, 0xD8, 0x98, 0xFD); + CALC_K (k, 12, 0x18, 0x37, 0xF7, 0x71); + CALC_K (k, 14, 0xEC, 0xF1, 0x6C, 0xE1); + CALC_K (k, 16, 0x43, 0x30, 0x75, 0x0F); + CALC_K (k, 18, 0x37, 0xF8, 0x26, 0x1B); + CALC_K (k, 20, 0xFA, 0x87, 0x13, 0xFA); + CALC_K (k, 22, 0x94, 0x06, 0x48, 0x3F); + CALC_K (k, 24, 0xF2, 0x5E, 0xD0, 0xBA); + CALC_K (k, 26, 0x8B, 0xAE, 0x30, 0x5B); + CALC_K (k, 28, 0x84, 0x8A, 0x54, 0x00); + CALC_K (k, 30, 0xDF, 0xBC, 0x23, 0x9D); + } + + return 0; +} + +EXPORT_SYMBOL_GPL(twofish_setkey); + +MODULE_LICENSE("GPL"); +MODULE_DESCRIPTION("Twofish cipher common functions"); diff --git a/drivers/crypto/Kconfig b/drivers/crypto/Kconfig index 4263935..3343c1f 100644 --- a/drivers/crypto/Kconfig +++ b/drivers/crypto/Kconfig @@ -3,21 +3,50 @@ menu "Hardware crypto devices" config CRYPTO_DEV_PADLOCK tristate "Support for VIA PadLock ACE" depends on CRYPTO && X86_32 + default m help Some VIA processors come with an integrated crypto engine (so called VIA PadLock ACE, Advanced Cryptography Engine) - that provides instructions for very fast {en,de}cryption - with some algorithms. + that provides instructions for very fast cryptographic + operations with supported algorithms. The instructions are used only when the CPU supports them. - Otherwise software encryption is used. If you are unsure, - say Y. + Otherwise software encryption is used. + + Selecting M for this option will compile a helper module + padlock.ko that should autoload all below configured + algorithms. Don't worry if your hardware does not support + some or all of them. In such case padlock.ko will + simply write a single line into the kernel log informing + about its failure but everything will keep working fine. + + If you are unsure, say M. The compiled module will be + called padlock.ko config CRYPTO_DEV_PADLOCK_AES - bool "Support for AES in VIA PadLock" + tristate "PadLock driver for AES algorithm" depends on CRYPTO_DEV_PADLOCK - default y + default m help Use VIA PadLock for AES algorithm. + Available in VIA C3 and newer CPUs. + + If unsure say M. The compiled module will be + called padlock-aes.ko + +config CRYPTO_DEV_PADLOCK_SHA + tristate "PadLock driver for SHA1 and SHA256 algorithms" + depends on CRYPTO_DEV_PADLOCK + select CRYPTO_SHA1 + select CRYPTO_SHA256 + default m + help + Use VIA PadLock for SHA1/SHA256 algorithms. + + Available in VIA C7 and newer processors. + + If unsure say M. The compiled module will be + called padlock-sha.ko + endmenu diff --git a/drivers/crypto/Makefile b/drivers/crypto/Makefile index 45426ca..4c3d0ec 100644 --- a/drivers/crypto/Makefile +++ b/drivers/crypto/Makefile @@ -1,7 +1,3 @@ - obj-$(CONFIG_CRYPTO_DEV_PADLOCK) += padlock.o - -padlock-objs-$(CONFIG_CRYPTO_DEV_PADLOCK_AES) += padlock-aes.o - -padlock-objs := padlock-generic.o $(padlock-objs-y) - +obj-$(CONFIG_CRYPTO_DEV_PADLOCK_AES) += padlock-aes.o +obj-$(CONFIG_CRYPTO_DEV_PADLOCK_SHA) += padlock-sha.o diff --git a/drivers/crypto/padlock-aes.c b/drivers/crypto/padlock-aes.c index b643d71..3e68370 100644 --- a/drivers/crypto/padlock-aes.c +++ b/drivers/crypto/padlock-aes.c @@ -59,6 +59,17 @@ #define AES_BLOCK_SIZE 16 /* ditto */ #define AES_EXTENDED_KEY_SIZE 64 /* in uint32_t units */ #define AES_EXTENDED_KEY_SIZE_B (AES_EXTENDED_KEY_SIZE * sizeof(uint32_t)) +/* Control word. */ +struct cword { + unsigned int __attribute__ ((__packed__)) + rounds:4, + algo:3, + keygen:1, + interm:1, + encdec:1, + ksize:2; +} __attribute__ ((__aligned__(PADLOCK_ALIGNMENT))); + /* Whenever making any changes to the following * structure *make sure* you keep E, d_data * and cword aligned on 16 Bytes boundaries!!! */ @@ -297,15 +308,16 @@ static inline struct aes_ctx *aes_ctx(st } static int aes_set_key(struct crypto_tfm *tfm, const u8 *in_key, - unsigned int key_len, u32 *flags) + unsigned int key_len) { struct aes_ctx *ctx = aes_ctx(tfm); const __le32 *key = (const __le32 *)in_key; + u32 *flags = &tfm->crt_flags; uint32_t i, t, u, v, w; uint32_t P[AES_EXTENDED_KEY_SIZE]; uint32_t rounds; - if (key_len != 16 && key_len != 24 && key_len != 32) { + if (key_len % 8) { *flags |= CRYPTO_TFM_RES_BAD_KEY_LEN; return -EINVAL; } @@ -473,7 +485,7 @@ static unsigned int aes_decrypt_cbc(cons static struct crypto_alg aes_alg = { .cra_name = "aes", .cra_driver_name = "aes-padlock", - .cra_priority = 300, + .cra_priority = PADLOCK_CRA_PRIORITY, .cra_flags = CRYPTO_ALG_TYPE_CIPHER, .cra_blocksize = AES_BLOCK_SIZE, .cra_ctxsize = sizeof(struct aes_ctx), @@ -495,15 +507,41 @@ static struct crypto_alg aes_alg = { } }; -int __init padlock_init_aes(void) +static int __init padlock_init(void) { - printk(KERN_NOTICE PFX "Using VIA PadLock ACE for AES algorithm.\n"); + int ret; + + if (!cpu_has_xcrypt) { + printk(KERN_ERR PFX "VIA PadLock not detected.\n"); + return -ENODEV; + } + + if (!cpu_has_xcrypt_enabled) { + printk(KERN_ERR PFX "VIA PadLock detected, but not enabled. Hmm, strange...\n"); + return -ENODEV; + } gen_tabs(); - return crypto_register_alg(&aes_alg); + if ((ret = crypto_register_alg(&aes_alg))) { + printk(KERN_ERR PFX "VIA PadLock AES initialization failed.\n"); + return ret; + } + + printk(KERN_NOTICE PFX "Using VIA PadLock ACE for AES algorithm.\n"); + + return ret; } -void __exit padlock_fini_aes(void) +static void __exit padlock_fini(void) { crypto_unregister_alg(&aes_alg); } + +module_init(padlock_init); +module_exit(padlock_fini); + +MODULE_DESCRIPTION("VIA PadLock AES algorithm support"); +MODULE_LICENSE("GPL"); +MODULE_AUTHOR("Michal Ludvig"); + +MODULE_ALIAS("aes-padlock"); diff --git a/drivers/crypto/padlock-generic.c b/drivers/crypto/padlock-generic.c deleted file mode 100644 index 18cf0e8..0000000 --- a/drivers/crypto/padlock-generic.c +++ /dev/null @@ -1,63 +0,0 @@ -/* - * Cryptographic API. - * - * Support for VIA PadLock hardware crypto engine. - * - * Copyright (c) 2004 Michal Ludvig - * - * This program is free software; you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation; either version 2 of the License, or - * (at your option) any later version. - */ - -#include -#include -#include -#include -#include -#include -#include "padlock.h" - -static int __init -padlock_init(void) -{ - int ret = -ENOSYS; - - if (!cpu_has_xcrypt) { - printk(KERN_ERR PFX "VIA PadLock not detected.\n"); - return -ENODEV; - } - - if (!cpu_has_xcrypt_enabled) { - printk(KERN_ERR PFX "VIA PadLock detected, but not enabled. Hmm, strange...\n"); - return -ENODEV; - } - -#ifdef CONFIG_CRYPTO_DEV_PADLOCK_AES - if ((ret = padlock_init_aes())) { - printk(KERN_ERR PFX "VIA PadLock AES initialization failed.\n"); - return ret; - } -#endif - - if (ret == -ENOSYS) - printk(KERN_ERR PFX "Hmm, VIA PadLock was compiled without any algorithm.\n"); - - return ret; -} - -static void __exit -padlock_fini(void) -{ -#ifdef CONFIG_CRYPTO_DEV_PADLOCK_AES - padlock_fini_aes(); -#endif -} - -module_init(padlock_init); -module_exit(padlock_fini); - -MODULE_DESCRIPTION("VIA PadLock crypto engine support."); -MODULE_LICENSE("Dual BSD/GPL"); -MODULE_AUTHOR("Michal Ludvig"); diff --git a/drivers/crypto/padlock-sha.c b/drivers/crypto/padlock-sha.c new file mode 100644 index 0000000..b028db6 --- /dev/null +++ b/drivers/crypto/padlock-sha.c @@ -0,0 +1,339 @@ +/* + * Cryptographic API. + * + * Support for VIA PadLock hardware crypto engine. + * + * Copyright (c) 2006 Michal Ludvig + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include "padlock.h" + +#define SHA1_DEFAULT_FALLBACK "sha1-generic" +#define SHA1_DIGEST_SIZE 20 +#define SHA1_HMAC_BLOCK_SIZE 64 + +#define SHA256_DEFAULT_FALLBACK "sha256-generic" +#define SHA256_DIGEST_SIZE 32 +#define SHA256_HMAC_BLOCK_SIZE 64 + +static char *sha1_fallback = SHA1_DEFAULT_FALLBACK; +static char *sha256_fallback = SHA256_DEFAULT_FALLBACK; + +module_param(sha1_fallback, charp, 0644); +module_param(sha256_fallback, charp, 0644); + +MODULE_PARM_DESC(sha1_fallback, "Fallback driver for SHA1. Default is " + SHA1_DEFAULT_FALLBACK); +MODULE_PARM_DESC(sha256_fallback, "Fallback driver for SHA256. Default is " + SHA256_DEFAULT_FALLBACK); + +struct padlock_sha_ctx { + char *data; + size_t used; + int bypass; + void (*f_sha_padlock)(const char *in, char *out, int count); + struct crypto_tfm *fallback_tfm; +}; + +static inline struct padlock_sha_ctx *ctx(struct crypto_tfm *tfm) +{ + return (struct padlock_sha_ctx *)(crypto_tfm_ctx(tfm)); +} + +/* We'll need aligned address on the stack */ +#define NEAREST_ALIGNED(ptr) \ + ((void *)ALIGN((size_t)(ptr), PADLOCK_ALIGNMENT)) + +static struct crypto_alg sha1_alg, sha256_alg; + +static void padlock_sha_bypass(struct crypto_tfm *tfm) +{ + if (ctx(tfm)->bypass) + return; + + BUG_ON(!ctx(tfm)->fallback_tfm); + + crypto_digest_init(ctx(tfm)->fallback_tfm); + if (ctx(tfm)->data && ctx(tfm)->used) { + struct scatterlist sg; + + sg_set_buf(&sg, ctx(tfm)->data, ctx(tfm)->used); + crypto_digest_update(ctx(tfm)->fallback_tfm, &sg, 1); + } + + ctx(tfm)->used = 0; + ctx(tfm)->bypass = 1; +} + +static void padlock_sha_init(struct crypto_tfm *tfm) +{ + ctx(tfm)->used = 0; + ctx(tfm)->bypass = 0; +} + +static void padlock_sha_update(struct crypto_tfm *tfm, + const uint8_t *data, unsigned int length) +{ + /* Our buffer is always one page. */ + if (unlikely(!ctx(tfm)->bypass && + (ctx(tfm)->used + length > PAGE_SIZE))) + padlock_sha_bypass(tfm); + + if (unlikely(ctx(tfm)->bypass)) { + struct scatterlist sg; + BUG_ON(!ctx(tfm)->fallback_tfm); + sg_set_buf(&sg, (uint8_t *)data, length); + crypto_digest_update(ctx(tfm)->fallback_tfm, &sg, 1); + return; + } + + memcpy(ctx(tfm)->data + ctx(tfm)->used, data, length); + ctx(tfm)->used += length; +} + +static inline void padlock_output_block(uint32_t *src, + uint32_t *dst, size_t count) +{ + while (count--) + *dst++ = swab32(*src++); +} + +static void padlock_do_sha1(const char *in, char *out, int count) +{ + /* We can't store directly to *out as it may be unaligned. */ + /* BTW Don't reduce the buffer size below 128 Bytes! + * PadLock microcode needs it that big. */ + char buf[128+16]; + char *result = NEAREST_ALIGNED(buf); + + ((uint32_t *)result)[0] = 0x67452301; + ((uint32_t *)result)[1] = 0xEFCDAB89; + ((uint32_t *)result)[2] = 0x98BADCFE; + ((uint32_t *)result)[3] = 0x10325476; + ((uint32_t *)result)[4] = 0xC3D2E1F0; + + asm volatile (".byte 0xf3,0x0f,0xa6,0xc8" /* rep xsha1 */ + : "+S"(in), "+D"(result) + : "c"(count), "a"(0)); + + padlock_output_block((uint32_t *)result, (uint32_t *)out, 5); +} + +static void padlock_do_sha256(const char *in, char *out, int count) +{ + /* We can't store directly to *out as it may be unaligned. */ + /* BTW Don't reduce the buffer size below 128 Bytes! + * PadLock microcode needs it that big. */ + char buf[128+16]; + char *result = NEAREST_ALIGNED(buf); + + ((uint32_t *)result)[0] = 0x6A09E667; + ((uint32_t *)result)[1] = 0xBB67AE85; + ((uint32_t *)result)[2] = 0x3C6EF372; + ((uint32_t *)result)[3] = 0xA54FF53A; + ((uint32_t *)result)[4] = 0x510E527F; + ((uint32_t *)result)[5] = 0x9B05688C; + ((uint32_t *)result)[6] = 0x1F83D9AB; + ((uint32_t *)result)[7] = 0x5BE0CD19; + + asm volatile (".byte 0xf3,0x0f,0xa6,0xd0" /* rep xsha256 */ + : "+S"(in), "+D"(result) + : "c"(count), "a"(0)); + + padlock_output_block((uint32_t *)result, (uint32_t *)out, 8); +} + +static void padlock_sha_final(struct crypto_tfm *tfm, uint8_t *out) +{ + if (unlikely(ctx(tfm)->bypass)) { + BUG_ON(!ctx(tfm)->fallback_tfm); + crypto_digest_final(ctx(tfm)->fallback_tfm, out); + ctx(tfm)->bypass = 0; + return; + } + + /* Pass the input buffer to PadLock microcode... */ + ctx(tfm)->f_sha_padlock(ctx(tfm)->data, out, ctx(tfm)->used); + + ctx(tfm)->used = 0; +} + +static int padlock_cra_init(struct crypto_tfm *tfm, const char *fallback_driver_name) +{ + /* For now we'll allocate one page. This + * could eventually be configurable one day. */ + ctx(tfm)->data = (char *)__get_free_page(GFP_KERNEL); + if (!ctx(tfm)->data) + return -ENOMEM; + + /* Allocate a fallback and abort if it failed. */ + ctx(tfm)->fallback_tfm = crypto_alloc_tfm(fallback_driver_name, 0); + if (!ctx(tfm)->fallback_tfm) { + printk(KERN_WARNING PFX "Fallback driver '%s' could not be loaded!\n", + fallback_driver_name); + free_page((unsigned long)(ctx(tfm)->data)); + return -ENOENT; + } + + return 0; +} + +static int padlock_sha1_cra_init(struct crypto_tfm *tfm) +{ + ctx(tfm)->f_sha_padlock = padlock_do_sha1; + + return padlock_cra_init(tfm, sha1_fallback); +} + +static int padlock_sha256_cra_init(struct crypto_tfm *tfm) +{ + ctx(tfm)->f_sha_padlock = padlock_do_sha256; + + return padlock_cra_init(tfm, sha256_fallback); +} + +static void padlock_cra_exit(struct crypto_tfm *tfm) +{ + if (ctx(tfm)->data) { + free_page((unsigned long)(ctx(tfm)->data)); + ctx(tfm)->data = NULL; + } + + BUG_ON(!ctx(tfm)->fallback_tfm); + crypto_free_tfm(ctx(tfm)->fallback_tfm); + ctx(tfm)->fallback_tfm = NULL; +} + +static struct crypto_alg sha1_alg = { + .cra_name = "sha1", + .cra_driver_name = "sha1-padlock", + .cra_priority = PADLOCK_CRA_PRIORITY, + .cra_flags = CRYPTO_ALG_TYPE_DIGEST, + .cra_blocksize = SHA1_HMAC_BLOCK_SIZE, + .cra_ctxsize = sizeof(struct padlock_sha_ctx), + .cra_module = THIS_MODULE, + .cra_list = LIST_HEAD_INIT(sha1_alg.cra_list), + .cra_init = padlock_sha1_cra_init, + .cra_exit = padlock_cra_exit, + .cra_u = { + .digest = { + .dia_digestsize = SHA1_DIGEST_SIZE, + .dia_init = padlock_sha_init, + .dia_update = padlock_sha_update, + .dia_final = padlock_sha_final, + } + } +}; + +static struct crypto_alg sha256_alg = { + .cra_name = "sha256", + .cra_driver_name = "sha256-padlock", + .cra_priority = PADLOCK_CRA_PRIORITY, + .cra_flags = CRYPTO_ALG_TYPE_DIGEST, + .cra_blocksize = SHA256_HMAC_BLOCK_SIZE, + .cra_ctxsize = sizeof(struct padlock_sha_ctx), + .cra_module = THIS_MODULE, + .cra_list = LIST_HEAD_INIT(sha256_alg.cra_list), + .cra_init = padlock_sha256_cra_init, + .cra_exit = padlock_cra_exit, + .cra_u = { + .digest = { + .dia_digestsize = SHA256_DIGEST_SIZE, + .dia_init = padlock_sha_init, + .dia_update = padlock_sha_update, + .dia_final = padlock_sha_final, + } + } +}; + +static void __init padlock_sha_check_fallbacks(void) +{ + struct crypto_tfm *tfm; + + /* We'll try to allocate one TFM for each fallback + * to test that the modules are available. */ + tfm = crypto_alloc_tfm(sha1_fallback, 0); + if (!tfm) { + printk(KERN_WARNING PFX "Couldn't load fallback module for '%s'. Tried '%s'.\n", + sha1_alg.cra_name, sha1_fallback); + } else { + printk(KERN_NOTICE PFX "Fallback for '%s' is driver '%s' (prio=%d)\n", sha1_alg.cra_name, + crypto_tfm_alg_driver_name(tfm), crypto_tfm_alg_priority(tfm)); + crypto_free_tfm(tfm); + } + + tfm = crypto_alloc_tfm(sha256_fallback, 0); + if (!tfm) { + printk(KERN_WARNING PFX "Couldn't load fallback module for '%s'. Tried '%s'.\n", + sha256_alg.cra_name, sha256_fallback); + } else { + printk(KERN_NOTICE PFX "Fallback for '%s' is driver '%s' (prio=%d)\n", sha256_alg.cra_name, + crypto_tfm_alg_driver_name(tfm), crypto_tfm_alg_priority(tfm)); + crypto_free_tfm(tfm); + } +} + +static int __init padlock_init(void) +{ + int rc = -ENODEV; + + if (!cpu_has_phe) { + printk(KERN_ERR PFX "VIA PadLock Hash Engine not detected.\n"); + return -ENODEV; + } + + if (!cpu_has_phe_enabled) { + printk(KERN_ERR PFX "VIA PadLock detected, but not enabled. Hmm, strange...\n"); + return -ENODEV; + } + + padlock_sha_check_fallbacks(); + + rc = crypto_register_alg(&sha1_alg); + if (rc) + goto out; + + rc = crypto_register_alg(&sha256_alg); + if (rc) + goto out_unreg1; + + printk(KERN_NOTICE PFX "Using VIA PadLock ACE for SHA1/SHA256 algorithms.\n"); + + return 0; + +out_unreg1: + crypto_unregister_alg(&sha1_alg); +out: + printk(KERN_ERR PFX "VIA PadLock SHA1/SHA256 initialization failed.\n"); + return rc; +} + +static void __exit padlock_fini(void) +{ + crypto_unregister_alg(&sha1_alg); + crypto_unregister_alg(&sha256_alg); +} + +module_init(padlock_init); +module_exit(padlock_fini); + +MODULE_DESCRIPTION("VIA PadLock SHA1/SHA256 algorithms support."); +MODULE_LICENSE("GPL"); +MODULE_AUTHOR("Michal Ludvig"); + +MODULE_ALIAS("sha1-padlock"); +MODULE_ALIAS("sha256-padlock"); diff --git a/drivers/crypto/padlock.c b/drivers/crypto/padlock.c new file mode 100644 index 0000000..ce58168 --- /dev/null +++ b/drivers/crypto/padlock.c @@ -0,0 +1,58 @@ +/* + * Cryptographic API. + * + * Support for VIA PadLock hardware crypto engine. + * + * Copyright (c) 2006 Michal Ludvig + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include "padlock.h" + +static int __init padlock_init(void) +{ + int success = 0; + + if (crypto_alg_available("aes-padlock", 0)) + success++; + + if (crypto_alg_available("sha1-padlock", 0)) + success++; + + if (crypto_alg_available("sha256-padlock", 0)) + success++; + + if (!success) { + printk(KERN_WARNING PFX "No VIA PadLock drivers have been loaded.\n"); + return -ENODEV; + } + + printk(KERN_NOTICE PFX "%d drivers are available.\n", success); + + return 0; +} + +static void __exit padlock_fini(void) +{ +} + +module_init(padlock_init); +module_exit(padlock_fini); + +MODULE_DESCRIPTION("Load all configured PadLock algorithms."); +MODULE_LICENSE("GPL"); +MODULE_AUTHOR("Michal Ludvig"); + diff --git a/drivers/crypto/padlock.h b/drivers/crypto/padlock.h index b78489b..7e3385b 100644 --- a/drivers/crypto/padlock.h +++ b/drivers/crypto/padlock.h @@ -15,22 +15,8 @@ #define _CRYPTO_PADLOCK_H #define PADLOCK_ALIGNMENT 16 -/* Control word. */ -struct cword { - unsigned int __attribute__ ((__packed__)) - rounds:4, - algo:3, - keygen:1, - interm:1, - encdec:1, - ksize:2; -} __attribute__ ((__aligned__(PADLOCK_ALIGNMENT))); - #define PFX "padlock: " -#ifdef CONFIG_CRYPTO_DEV_PADLOCK_AES -int padlock_init_aes(void); -void padlock_fini_aes(void); -#endif +#define PADLOCK_CRA_PRIORITY 300 #endif /* _CRYPTO_PADLOCK_H */ diff --git a/include/crypto/twofish.h b/include/crypto/twofish.h new file mode 100644 index 0000000..c408522 --- /dev/null +++ b/include/crypto/twofish.h @@ -0,0 +1,22 @@ +#ifndef _CRYPTO_TWOFISH_H +#define _CRYPTO_TWOFISH_H + +#include + +#define TF_MIN_KEY_SIZE 16 +#define TF_MAX_KEY_SIZE 32 +#define TF_BLOCK_SIZE 16 + +struct crypto_tfm; + +/* Structure for an expanded Twofish key. s contains the key-dependent + * S-boxes composed with the MDS matrix; w contains the eight "whitening" + * subkeys, K[0] through K[7]. k holds the remaining, "round" subkeys. Note + * that k[i] corresponds to what the Twofish paper calls K[i+8]. */ +struct twofish_ctx { + u32 s[4][256], w[8], k[32]; +}; + +int twofish_setkey(struct crypto_tfm *tfm, const u8 *key, unsigned int key_len); + +#endif diff --git a/include/linux/crypto.h b/include/linux/crypto.h index 7f94624..9d8bd3f 100644 --- a/include/linux/crypto.h +++ b/include/linux/crypto.h @@ -22,16 +22,21 @@ #include #include #include #include +#include #include /* * Algorithm masks and types. */ -#define CRYPTO_ALG_TYPE_MASK 0x000000ff +#define CRYPTO_ALG_TYPE_MASK 0x0000000f #define CRYPTO_ALG_TYPE_CIPHER 0x00000001 #define CRYPTO_ALG_TYPE_DIGEST 0x00000002 #define CRYPTO_ALG_TYPE_COMPRESS 0x00000004 +#define CRYPTO_ALG_LARVAL 0x00000010 +#define CRYPTO_ALG_DEAD 0x00000020 +#define CRYPTO_ALG_DYING 0x00000040 + /* * Transform masks and values (for crt_flags). */ @@ -80,7 +85,7 @@ struct cipher_alg { unsigned int cia_min_keysize; unsigned int cia_max_keysize; int (*cia_setkey)(struct crypto_tfm *tfm, const u8 *key, - unsigned int keylen, u32 *flags); + unsigned int keylen); void (*cia_encrypt)(struct crypto_tfm *tfm, u8 *dst, const u8 *src); void (*cia_decrypt)(struct crypto_tfm *tfm, u8 *dst, const u8 *src); @@ -103,9 +108,11 @@ struct digest_alg { void (*dia_init)(struct crypto_tfm *tfm); void (*dia_update)(struct crypto_tfm *tfm, const u8 *data, unsigned int len); + void (*dia_update_sg)(struct crypto_tfm *tfm, struct scatterlist *sg, + unsigned int nsg); void (*dia_final)(struct crypto_tfm *tfm, u8 *out); int (*dia_setkey)(struct crypto_tfm *tfm, const u8 *key, - unsigned int keylen, u32 *flags); + unsigned int keylen); }; struct compress_alg { @@ -121,12 +128,15 @@ #define cra_compress cra_u.compress struct crypto_alg { struct list_head cra_list; + struct list_head cra_users; + u32 cra_flags; unsigned int cra_blocksize; unsigned int cra_ctxsize; unsigned int cra_alignmask; int cra_priority; + atomic_t cra_refcnt; char cra_name[CRYPTO_MAX_ALG_NAME]; char cra_driver_name[CRYPTO_MAX_ALG_NAME]; @@ -139,6 +149,7 @@ struct crypto_alg { int (*cra_init)(struct crypto_tfm *tfm); void (*cra_exit)(struct crypto_tfm *tfm); + void (*cra_destroy)(struct crypto_alg *alg); struct module *cra_module; }; @@ -201,9 +212,6 @@ struct digest_tfm { unsigned int nsg, u8 *out); int (*dit_setkey)(struct crypto_tfm *tfm, const u8 *key, unsigned int keylen); -#ifdef CONFIG_CRYPTO_HMAC - void *dit_hmac_block; -#endif }; struct compress_tfm { @@ -234,6 +242,15 @@ struct crypto_tfm { char __crt_ctx[] __attribute__ ((__aligned__)); }; +enum { + CRYPTOA_UNSPEC, + CRYPTOA_ALG, +}; + +struct crypto_attr_alg { + char name[CRYPTO_MAX_ALG_NAME]; +}; + /* * Transform user interface. */ @@ -258,6 +275,16 @@ static inline const char *crypto_tfm_alg return tfm->__crt_alg->cra_name; } +static inline const char *crypto_tfm_alg_driver_name(struct crypto_tfm *tfm) +{ + return tfm->__crt_alg->cra_driver_name; +} + +static inline int crypto_tfm_alg_priority(struct crypto_tfm *tfm) +{ + return tfm->__crt_alg->cra_priority; +} + static inline const char *crypto_tfm_alg_modname(struct crypto_tfm *tfm) { return module_name(tfm->__crt_alg->cra_module); @@ -313,6 +340,16 @@ static inline unsigned int crypto_tfm_ct return __alignof__(tfm->__crt_ctx); } +static inline void *crypto_tfm_ctx_aligned(struct crypto_tfm *tfm) +{ + unsigned long addr = (unsigned long)crypto_tfm_ctx(tfm); + unsigned long align = crypto_tfm_alg_alignmask(tfm); + + if (align <= crypto_tfm_ctx_alignment()) + align = 1; + return (void *)ALIGN(addr, align); +} + /* * API wrappers. */ @@ -348,8 +385,6 @@ static inline int crypto_digest_setkey(s const u8 *key, unsigned int keylen) { BUG_ON(crypto_tfm_alg_type(tfm) != CRYPTO_ALG_TYPE_DIGEST); - if (tfm->crt_digest.dit_setkey == NULL) - return -ENOSYS; return tfm->crt_digest.dit_setkey(tfm, key, keylen); } @@ -428,18 +463,5 @@ static inline int crypto_comp_decompress return tfm->crt_compress.cot_decompress(tfm, src, slen, dst, dlen); } -/* - * HMAC support. - */ -#ifdef CONFIG_CRYPTO_HMAC -void crypto_hmac_init(struct crypto_tfm *tfm, u8 *key, unsigned int *keylen); -void crypto_hmac_update(struct crypto_tfm *tfm, - struct scatterlist *sg, unsigned int nsg); -void crypto_hmac_final(struct crypto_tfm *tfm, u8 *key, - unsigned int *keylen, u8 *out); -void crypto_hmac(struct crypto_tfm *tfm, u8 *key, unsigned int *keylen, - struct scatterlist *sg, unsigned int nsg, u8 *out); -#endif /* CONFIG_CRYPTO_HMAC */ - #endif /* _LINUX_CRYPTO_H */ diff --git a/include/net/ah.h b/include/net/ah.h index ceff00a..67e9646 100644 --- a/include/net/ah.h +++ b/include/net/ah.h @@ -26,9 +26,10 @@ ah_hmac_digest(struct ah_data *ahp, stru struct crypto_tfm *tfm = ahp->tfm; memset(auth_data, 0, ahp->icv_trunc_len); - crypto_hmac_init(tfm, ahp->key, &ahp->key_len); - skb_icv_walk(skb, tfm, 0, skb->len, crypto_hmac_update); - crypto_hmac_final(tfm, ahp->key, &ahp->key_len, ahp->work_icv); + crypto_digest_setkey(tfm, ahp->key, ahp->key_len); + crypto_digest_init(tfm); + skb_icv_walk(skb, tfm, 0, skb->len, crypto_digest_update); + crypto_digest_final(tfm, ahp->work_icv); memcpy(auth_data, ahp->work_icv, ahp->icv_trunc_len); } diff --git a/include/net/esp.h b/include/net/esp.h index 90cd94f..2ec3016 100644 --- a/include/net/esp.h +++ b/include/net/esp.h @@ -50,9 +50,10 @@ esp_hmac_digest(struct esp_data *esp, st char *icv = esp->auth.work_icv; memset(auth_data, 0, esp->auth.icv_trunc_len); - crypto_hmac_init(tfm, esp->auth.key, &esp->auth.key_len); - skb_icv_walk(skb, tfm, offset, len, crypto_hmac_update); - crypto_hmac_final(tfm, esp->auth.key, &esp->auth.key_len, icv); + crypto_digest_setkey(tfm, esp->auth.key, esp->auth.key_len); + crypto_digest_init(tfm); + skb_icv_walk(skb, tfm, offset, len, crypto_digest_update); + crypto_digest_final(tfm, icv); memcpy(auth_data, icv, esp->auth.icv_trunc_len); } diff --git a/include/net/sctp/constants.h b/include/net/sctp/constants.h index c51541e..57166bf 100644 --- a/include/net/sctp/constants.h +++ b/include/net/sctp/constants.h @@ -312,9 +312,9 @@ #define SCTP_COOKIE_MULTIPLE 32 /* Pad o */ #if defined (CONFIG_SCTP_HMAC_MD5) -#define SCTP_COOKIE_HMAC_ALG "md5" +#define SCTP_COOKIE_HMAC_ALG "hmac(md5)" #elif defined (CONFIG_SCTP_HMAC_SHA1) -#define SCTP_COOKIE_HMAC_ALG "sha1" +#define SCTP_COOKIE_HMAC_ALG "hmac(sha1)" #else #define SCTP_COOKIE_HMAC_ALG NULL #endif diff --git a/include/net/sctp/sctp.h b/include/net/sctp/sctp.h index a9663b4..4b6ed61 100644 --- a/include/net/sctp/sctp.h +++ b/include/net/sctp/sctp.h @@ -330,17 +330,6 @@ static inline void sctp_v6_exit(void) { #endif /* #if defined(CONFIG_IPV6) */ -/* Some wrappers, in case crypto not available. */ -#if defined (CONFIG_CRYPTO_HMAC) -#define sctp_crypto_alloc_tfm crypto_alloc_tfm -#define sctp_crypto_free_tfm crypto_free_tfm -#define sctp_crypto_hmac crypto_hmac -#else -#define sctp_crypto_alloc_tfm(x...) NULL -#define sctp_crypto_free_tfm(x...) -#define sctp_crypto_hmac(x...) -#endif - /* Map an association to an assoc_id. */ static inline sctp_assoc_t sctp_assoc2id(const struct sctp_association *asoc) diff --git a/include/net/xfrm.h b/include/net/xfrm.h index 9c5ee9f..2f02624 100644 --- a/include/net/xfrm.h +++ b/include/net/xfrm.h @@ -855,6 +855,7 @@ struct xfrm_algo_comp_info { struct xfrm_algo_desc { char *name; + char *compat; u8 available:1; union { struct xfrm_algo_auth_info auth; diff --git a/net/sctp/endpointola.c b/net/sctp/endpointola.c index 67bd530..25d18f2 100644 --- a/net/sctp/endpointola.c +++ b/net/sctp/endpointola.c @@ -172,7 +172,7 @@ static void sctp_endpoint_destroy(struct sctp_unhash_endpoint(ep); /* Free up the HMAC transform. */ - sctp_crypto_free_tfm(sctp_sk(ep->base.sk)->hmac); + crypto_free_tfm(sctp_sk(ep->base.sk)->hmac); /* Cleanup. */ sctp_inq_free(&ep->base.inqueue); diff --git a/net/sctp/sm_make_chunk.c b/net/sctp/sm_make_chunk.c index 2a87736..25a3894 100644 --- a/net/sctp/sm_make_chunk.c +++ b/net/sctp/sm_make_chunk.c @@ -1340,8 +1340,9 @@ static sctp_cookie_param_t *sctp_pack_co keylen = SCTP_SECRET_SIZE; key = (char *)ep->secret_key[ep->current_key]; - sctp_crypto_hmac(sctp_sk(ep->base.sk)->hmac, key, &keylen, - &sg, 1, cookie->signature); + crypto_digest_setkey(sctp_sk(ep->base.sk)->hmac, key, keylen); + crypto_digest_digest(sctp_sk(ep->base.sk)->hmac, &sg, 1, + cookie->signature); } nodata: @@ -1403,15 +1404,16 @@ struct sctp_association *sctp_unpack_coo key = (char *)ep->secret_key[ep->current_key]; memset(digest, 0x00, SCTP_SIGNATURE_SIZE); - sctp_crypto_hmac(sctp_sk(ep->base.sk)->hmac, key, &keylen, &sg, - 1, digest); + crypto_digest_setkey(sctp_sk(ep->base.sk)->hmac, key, keylen); + crypto_digest_digest(sctp_sk(ep->base.sk)->hmac, &sg, 1, digest); if (memcmp(digest, cookie->signature, SCTP_SIGNATURE_SIZE)) { /* Try the previous key. */ key = (char *)ep->secret_key[ep->last_key]; memset(digest, 0x00, SCTP_SIGNATURE_SIZE); - sctp_crypto_hmac(sctp_sk(ep->base.sk)->hmac, key, &keylen, - &sg, 1, digest); + crypto_digest_setkey(sctp_sk(ep->base.sk)->hmac, key, keylen); + crypto_digest_digest(sctp_sk(ep->base.sk)->hmac, &sg, 1, + digest); if (memcmp(digest, cookie->signature, SCTP_SIGNATURE_SIZE)) { /* Yikes! Still bad signature! */ diff --git a/net/sctp/socket.c b/net/sctp/socket.c index 0a2c71d..f162e86 100644 --- a/net/sctp/socket.c +++ b/net/sctp/socket.c @@ -4855,7 +4855,7 @@ int sctp_inet_listen(struct socket *sock /* Allocate HMAC for generating cookie. */ if (sctp_hmac_alg) { - tfm = sctp_crypto_alloc_tfm(sctp_hmac_alg, 0); + tfm = crypto_alloc_tfm(sctp_hmac_alg, 0); if (!tfm) { err = -ENOSYS; goto out; @@ -4881,7 +4881,7 @@ out: sctp_release_sock(sk); return err; cleanup: - sctp_crypto_free_tfm(tfm); + crypto_free_tfm(tfm); goto out; } diff --git a/net/xfrm/xfrm_algo.c b/net/xfrm/xfrm_algo.c index 04e1aea..2783d4e 100644 --- a/net/xfrm/xfrm_algo.c +++ b/net/xfrm/xfrm_algo.c @@ -30,7 +30,8 @@ #include */ static struct xfrm_algo_desc aalg_list[] = { { - .name = "digest_null", + .name = "hmac(digest_null)", + .compat = "digest_null", .uinfo = { .auth = { @@ -47,7 +48,8 @@ static struct xfrm_algo_desc aalg_list[] } }, { - .name = "md5", + .name = "hmac(md5)", + .compat = "md5", .uinfo = { .auth = { @@ -64,7 +66,8 @@ static struct xfrm_algo_desc aalg_list[] } }, { - .name = "sha1", + .name = "hmac(sha1)", + .compat = "sha1", .uinfo = { .auth = { @@ -81,7 +84,8 @@ static struct xfrm_algo_desc aalg_list[] } }, { - .name = "sha256", + .name = "hmac(sha256)", + .compat = "sha256", .uinfo = { .auth = { @@ -98,7 +102,8 @@ static struct xfrm_algo_desc aalg_list[] } }, { - .name = "ripemd160", + .name = "hmac(ripemd160)", + .compat = "ripemd160", .uinfo = { .auth = { @@ -359,7 +364,8 @@ static struct xfrm_algo_desc *xfrm_get_b return NULL; for (i = 0; i < entries; i++) { - if (strcmp(name, list[i].name)) + if (strcmp(name, list[i].name) && + (!list[i].compat || strcmp(name, list[i].compat))) continue; if (list[i].available) diff --git a/net/xfrm/xfrm_user.c b/net/xfrm/xfrm_user.c index 3e6a722..444658d 100644 --- a/net/xfrm/xfrm_user.c +++ b/net/xfrm/xfrm_user.c @@ -212,6 +212,7 @@ static int attach_one_algo(struct xfrm_a return -ENOMEM; memcpy(p, ualg, len); + strcpy(p->alg_name, algo->name); *algpp = p; return 0; }