Don't generate the list
It is usually a bad idea to try to generate a list of all permutations as the length of that list can be very large for some lists. E.g., a list of the permutations of a list of length ten will be 10! = 3,628,800. What is generally a better idea is to iterate over the permutations one at a time without ever generating them all into a list. This is sometimes referred to as Visiting the permutations with a visitor function.Even without accumulating a list of permutations, assuming 1 microsecond per iteration, iterating over the permutations of a list of 24 elements will take longer than the age of the universe. This is assuming the universe is 15 billion years old, which is between 23! and 24! microseconds old. Please someone correct me in the comments below if my off the cuff calculation is wrong.
In this blog we present ways to implement the
function map_permutations which can be called as
follows: (map_permutations VISITOR DATA),
where VISITOR is a unary function which will be called on
each permutation, and DATA is a list of items.
Brute force approach
The following implementation ofmap_permutations_BF is
more or less a recursive function which implements N
consecutive foreach loops, where N is the length of the
given list. Each step, 0<=k<=N, of the recursion makes N-k calls to, the
local function next. Thus, next reaches the
k=N level N! times which it has each time accumulated one permutation.(defun map_permutations_BF (visit data "ul")
(let ((N (length data)))
(labels ((next (so_far k)
(cond
((eqv k N)
(visit (mapcar car so_far)))
(t
k++
(foreach map items data
(unless (memq items so_far)
(next (cons items so_far) k)))))))
(next nil 0))))
The example above is SKILL++ code so as always with SKILL++, you need
to save it in a file with a .ils extension, or wrap it
in (inScheme ...) if pasting into the CIW window.
Example: printing the permutations
Here is a few of examples of how to usemap_permutations_BF. The first example prints each
permutation of (1 2 3 4) to standard output.(map_permutations_BF println'(1 2 3 4 5))Here is the standard out of evaluating that expression:
(4 3 2 1) (3 4 2 1) (4 2 3 1) (2 4 3 1) (3 2 4 1) (2 3 4 1) (4 3 1 2) (3 4 1 2) (4 1 3 2) (1 4 3 2) (3 1 4 2) (1 3 4 2) (4 2 1 3) (2 4 1 3) (4 1 2 3) (1 4 2 3) (2 1 4 3) (1 2 4 3) (3 2 1 4) (2 3 1 4) (3 1 2 4) (1 3 2 4) (2 1 3 4) (1 2 3 4)
Counting the permutations
The second example counts the number of permutations of(1 2 3 4 5 6 7 8 9):(let ((c 0)) (map_permutations_BF (lambda (_) c++)'(1 2 3 4 5 6 7 8 9)) c)==> 362880
Collecting a list of permutations
The third example collects and returns the list of permutations of(1 2 3):(let (data) (map_permutations_BF (lambda (p) (push p data))'(1 2 3)) data)==> ((1 2 3) (2 1 3) (1 3 2) (3 1 2) (2 3 1) (3 2 1))
Trying different algorithms
A very good reference source for discussions and implementations of algorithms in general is Donald Knuth's bookArt of Computer Programming. In particular Volume 4 Fascicle 2 discusses how to generate permutations. The following are three different algorithms implemented from that fascicle.
PLEASE NOTE that I have not tried to improve the readability of the algorithms, but rather I've tried to keep the SKILL code as close as possible to Knuths pseudo-code from the book. I would advise against trying to improve the code. One great advantage of keeping the code as it is, is that you know it works. Another argument for keeping it as it is, and resist improving is that if someone later comes along and reads your code, they can read the referenced pages of Art of Computer Programming to discover the justifications for the implementation as well as discussion of limitations and performance.
Quick review of SKILL prog/go
As with most of the algorithms in Art of Computer Programming,
these algorithms are highly optimized and depend on GOTO.
The GOTO operator in SKILL is called go and is
only available inside (prog ...) within traditional
SKILL. Unfortunately, prog/go is NOT
available in SKILL++. This means unless you want to convert all
the GOTOs in Knuth's code to some other style, you'll have to
use .il as your file name suffix when implementing these
functions, or of course you may wrap the code in (inSkill
...).
A moral of this story is that sometimes SKILL++ is good for solving a problem, and sometimes traditional SKILL is better. In the SKILL language you have both together. You can use the dialect which works best for your problem space, or you can mix and match as necessary as your application demands.
The SKILL prog allows its body to
call return with an argument which causes
the prog for to exit and to return the argument which was
given to return. For example:
(prog () (return 42) (println 'hello))==> 42In the previous example, the
(prog ...) form returns 42,
and the line (println 'hello) is never reached. If while
executing the body of the (prog ...) form,
no (return ...) is encountered, the (prog
...) form returns nil.
Another Knuth-friendly feature of SKILL prog but
not the SKILL++ prog is that it allows top-level labels
which are valid unquoted operands for the go operator.
In the following
example, L0, L1, L2,
and L3 are labels. Some of these are used by (go
..); in particular (go L1), (go L2),
and (go L0).
(prog (n)
(go L1)
L0
(printf "finished\n")
(return 'done)
L1
n = 0
L2
(printf "n=")
(println n)
L3
(if (n < 12)
then n++
(go L2)
else (go L0))
(printf 'never_reached))
You'll find this prog/go construct useful when
transcribing Knuth's algorithms.Plain changes
The following implements algorithm P (Plain changes) from Donald E. Knuth, The Art of Computer Programming Volume 4 Fascicle 2, Generating All Tuples and Permutations, page 42.(defun map_permutations_P (visit data "ul")
(letseq ((n (length data))
(a (listToVector (cons nil data)))
(c (makeVector (add1 n)))
(o (makeVector (add1 n)))
j
s
q)
(prog ()
P1 ; Initialize
(for j 1 n
c[j] = 0
o[j] = 1)
P2 ; Visit
(funcall visit (cdr (vectorToList a)))
P3 ; Prepare for change.
j = n
s = 0
P4 ; Ready to change?
q = c[j] + o[j]
(cond
((q < 0)
(go P7))
((q == j)
(go P6)))
P5 ; Change
(let ((u j-c[j]+s)
(v j-q+s));; swap a[v] with a[u]
a[v] = (prog1 a[u]
a[u] = a[v]))
c[j] = q
(go P2)
P6 ; Increase s
(if (j == 1)
(return)
(s++))
P7 ; Switch direction
o[j] = -o[j]
j--
(go P4))))Converting Knuth algorithms to SKILL
List vs Array In SKILL you normally work with lists, and consequently you might want to visit the permutations of a list. The Knuth algorithms in this article visits the permutation of a fixed size array. Because of this, these implementations ofmap_permutations expect
lists as input and output, while internally use arrays in-keeping with
the Knuth algorithm. Converting between list and array is done
using listToVector and vectorToList.Zero indexed vs. one indexed SKILL only provided zero-based arrays. Some of the Knuth algorithms use one-based and some used zero-based arrays. Where appropriate, filler elements are used for the 0th element when the Knuth array is one-based. You'll see expressions such as
(funcall visit (cdr (vectorToList a))) in the SKILL
implementations, which means the corresponding array is one-based in
the Knuth algorithm, so we have to discard the 0th element when
passing the data to the visit function. The following
table summarizes how to
use makeVector, vectorToList,
and listToVector with zero-indexed and one-indexed
arrays.
| allocate | array to list | list to array | |
|---|---|---|---|
| N element zero-indexed | (makeVector N) | (vectorToList myarr) | (listToVector mylist) |
| N element one-indexed | (makeVector N+1) | (cdr (vectorToList myarr)) | (listToVector (cons nil mylist)) |
Use of
prog/go/return
As described above the use of labels and GOTO in the
Knuth algorithms is easily implemented using the SKILL (prog
...) construct.Use of
do/until
Some Knuth algorithms use do/until,
which is available in SKILL++ but not in SKILL. We solve this problem
by using the following do_until macro, which takes a test
expression as first argument, and a body of expressions. The effect
is that the body of expressions is evaluated once, and thereafter it
is evaluated again and again as long as re-evaluating the tests
returns nil.(defmacro do_until (test @rest body)
`(progn ,@body
(while (not ,test)
,@body)))Swapping elements Swapping two elements, A and B, can be done as followings, but when this is done an appropriate comment is added.
A = (prog1 B
B = A)Ehrlich swaps
The following implements algorithm E (Ehrlich swaps) from Donald E. Knuth, The Art of Computer Programming Volume 4 Fascicle 2, Generating All Tuples and Permutations, page 57. According to Gideon Ehrlich, the original author of this algorithm, the most amazing thing about it is that it works.(defun map_permutations_E (visit data "ul")
(letseq ((a (listToVector data))
(n (length data))
(b (makeVector n))
(c (makeVector n+1))
k
j)
(prog ()
E1 ; Initialize
(for j 0 n-1
b[j] = j
c[j+1] = 0)
E2 ; Visit
(funcall visit (vectorToList a))
E3 ; Find k
k = 1
(when (c[k] == k)
(do_until (c[k] < k)
c[k] = 0
k++))
(if (k == n)
(return)
c[k] = c[k] + 1)
E4 ; Swap
a[0] = (prog1 a[b[k]]
a[b[k]] = a[0])
E5 ; Flip
j = 1
k--
(while (j < k);; swap b[j] and b[k]
b[j] = (prog1 b[k]
b[k] = b[j])
j++
k--)
(go E2))))Permutation generation by cyclic shifts
The following implements algorithm C (Permutation generation by cyclic shifts) from Donald E. Knuth, The Art of Computer Programming Volume 4 Fascicle 2, Generating All Tuples and Permutations, page 56.(defun map_permutations_C (visit data "ul")
(letseq ((x (listToVector (cons nil data)))
(n (length data))
a
k
j)
(prog ()
C1 ; Initialize
a = (listToVector (cons nil data))
C2 ; Visit
(funcall visit (cdr (vectorToList a)))
k = n
C3 ; Shift;; (a1 a2 a3 ..ak) <-- (a2 a3 ...ak a1)
a[k] = (prog1 a[1]
(for j 1 k-1
a[j] = a[j+1]))
(when (a[k] != x[k])
(go C2))
C4 ; Decrease k.
k--
(when (k > 1)
(go C3)))))Summary
In this article we looked at several things.- Four functions for visiting all permutations of a given list.
Feel free to copy and paste these implementations into your SKILL
programs, but please remember to give credit to Donald Knuth where
appropriate.
map_permutations_BF-- Brute forcemap_permutations_C-- Cyclic shiftsmap_permutations_P-- Plain changesmap_permutations_E-- Ehrlich swaps
- Translating algorithms from Donald Knuth's Art of Computer Programming to SKILL
- Seldom used features of SKILL
(prog ...)such as(go ...) - Examples using
makeVector,listToVector, andvectorToList.
See Also
- The Art of Computer Programming by Donald Knuth.
- Donald Knuth
- Age of the Universe
- KXCD
