SKILL for the Skilled: How to Copy a Hash Table

In this posting I want to look at ways to copy a hash table in SKILL. There are several ways you might naively try to do this, but some of these naive approaches have gotchas which you should be aware of.

In the following paragraphs several inferior functions will be presented: portable_1, copyTable_2, copyTable_3, copyTable_4, and copyTable_5. Finally three useful robust functions will be presented (copyTable, getHashDefault, and getHashPrintName)which you, the reader, can copy (attach a customer specific prefix to) and use in your own programs.

Copy by iteration

The first way you might try to implement the function copyTable is as follows.

(defun copyTable_1 (hash)
  (let ((new (makeTable 'copy)))
    (foreach key hash
      new[key] = hash[key])
    new))

The function copyTable_1 works perfectly for most applications for which you might want to use it. This approach, unfortunately, has several limiting corner cases which might be important.

Troublesome hash keys

In some sense this example reminds me of little Bobby Tables from xkcd.

The problem is that if you are not in control of the data, then you don't know what might be in there.

If the function copyTable_1 is defined as shown above, it will fail if either of the following keys are found in the hash table: ?, ??, or unbound. While it is indeed unlikely that most people would ever encounter this case, it is nevertheless something to consider if you want your implementation of copyTable which is robust and general purpose.

Try the following test case and see what happens.

(let ((table (makeTable 'x)))
  table[?] = 1
  table[??] = 2
  table['unbound] = 3 
  (copyTable_1 table))*Error* eval: unbound variable - key<<< Stack Trace >>>
(new[key] = hash[key])
foreach(key hash (new[key] = hash[key]))
let(((new makeTable(&))) foreach(key hash (new[key] = hash[key])) new)
copyTable_1(table)
let(((table makeTable(&))) (table[?] = 1) (table[??] = 2) (table['unbound] = 
   3) copyTable_1(table))

Using the unbound is easier in SKILL++

The resulting stack-trace is admittedly confusing.

The error is triggered because foreach encounters the symbol unbound in the hash table. On doing so, it effectively sets the variable key to unbound, as if by (key = 'unbound). Thereafter new[key] contains a reference to an unbound variable. In SKILL, an error is triggered if an attempt is made to evaluate a variable whose value is the symbol: unbound.

To demonstrate this to yourself, try the following in SKILL.

(inSkill
  (let ((key 'unbound))
    (printf "the value is [%L]\n" key)))
*Error* eval: unbound variable - key<<< Stack Trace >>>
printf("the value is <%L>\n" key)
let(((key 'unbound)) printf("the value is [%L]\n" key))
inSkill(let(((key &)) printf("the value is [%L]\n" key)))

This problem is easily avoided by using SKILL++. In SKILL++ the unbound symbol is not special: it is a supported feature to set a variable to the symbol unbound, and to evaluate the variable thereafter. Try the same example as above, but using in inScheme rather than inSkill.

(inScheme
  (let ((key 'unbound))
    (printf "the value is [%L]\n" key)))
the value is [unbound]

Redefine the function in SKILL++

We can fix the problem associated with the unbound symbol simply by defining the function either in a .ils file or by surrounding the definition with (inScheme ...).

(inScheme
  (defun copyTable_2 (hash)
    (let ((new (makeTable 'copy)))
      (foreach key hash
        new[key] = hash[key])
      new)))

Now if we run the same test again we see a different result.

(let ((table (makeTable 'x)))
  table[?] = 1
  table[??] = 2
  table['unbound] = 3 
  (copyTable_2 table))
table:copy

More troublesome hash keys: ? or ??

If we test a bit further we find that it has failed for another subtle reason. We can use the tableToList function to examine all the key/value pairs in a hash table. Notice that the content of the original list and the content of the copy are not the same.

(let ((table (makeTable 'x)))
  table[?] = 1
  table[??] = 2
  table['unbound] = 3

  (printf "contents of original:\n--> %L\n"
          (tableToList table))
  (printf "contents of copy:\n--> %L\n"
          (tableToList (copyTable_2 table))))
contents of original:
--> ((unbound 3) (? 1) (?? 2))
contents of copy:
--> ((unbound 3) (? (unbound ? ??)) (?? (unbound 3 ? 1 ?? 2)))

This is not something you normally need to worry about, because if you created the hash table in your SKILL application, then you probably know that neither ? nor ?? is a hash key in your table. However, if the goal is to write a general purpose function for copying any given hash table, then this is an exotic case you must consider.

Why the crazy behavior?

It is not so difficult to understand this crazy behavior. It is because ? and ?? have special meanings to the functions arrayref, get, getq and a few more functions. The following expressions do not retrieve the value of the hash key ?. Rather they all return the list of hash keys.

hash->?
hash[?]
(arrayref hash ?)
(get hash ?)
(getq hash ?)

Similarly hash->?? does not retrieve the value associated with the key ??, it instead returns a special list which embeds they hash keys and values.

This means the line new[key] = hash[key] within the previous versions of copyTable does something special with the value of the variable key is ? (or ??). It does the following: new[?] = hash[?], which assigns the ? key of the new hash table to the list of keys in the old hash table.

Use append with hash tables

To fix this problem, take advantage of the feature of the append SKILL function. It can append two hash tables.

(defun copyTable_3 (hash)
  (let ((new (makeTable 'copy)))
    (append new hash)
    new))

But since, append returns its first argument if the first argument is a hash table, copyTable_3 can be made much simpler.

(defun copyTable_4 (hash)
  (append (makeTable 'copy) 
          hash))

The append function does not have a problem when it encounters the symbols ?, ??, or unbound. Take a look at the same example using copyTable_4.

(let ((table (makeTable 'x)))
  table[?] = 1
  table[??] = 2
  table['unbound] = 3

  (printf "contents of original:\n-->%L\n"
          (tableToList table))
  (printf "contents of copy:\n-->%L\n"
          (tableToList (copyTable_4 table))))
contents of original:
-->((unbound 3) (? 1) (?? 2))
contents of copy:
-->((unbound 3) (? 1) (?? 2))

Even if you are not worried about the special, extremely unlikely hash keys discussed above, using append to copy hash tables makes for smaller and faster code.

Preserving the table name

Depending on how good a copy you need, you might also want the new table to print the same as the original table. Look at this example. One prints as table:data and the other as table:copy.

(let ((table (makeTable 'data)))
  (printf "original: %L\n" table)
  (printf "copy:     %L\n" (copyTable_4 table)))
original: table:data
copy:     table:copy

To fix this problem we need to pass the correct value to makeTable within the copyTable function as follows.

(defun copyTable_5 (hash)
  (append (makeTable (getHashPrintName hash))
          hash))

Here is an implementation of the function getHashPrintName which returns the print name of the given hash table. If the table prints as table:myname, getHashPrintName returns the string "myname".

(defun getHashPrintName (hash)
  (substring (sprintf nil "%L" hash)
             7))

If we test copyTable_5 we see that the two tables do print the same way.

(let ((table (makeTable 'data)))
  (printf "original: %L\n" table)
  (printf "copy:     %L\n" (copyTable_5 table)))
original: table:data
copy:     table:data

Preserving the default value

There is one more important difference between the two hash tables (the one given to copyTable_5 and the one it returns). The difference is how the hash tables behave when accessing a missing key. Take a look at this example.

(letseq ((table (makeTable 'data 0))
         (copy (copyTable_5 table)))
  (printf "default from original: %L\n" table[42])
  (printf "default from copy:     %L\n" copy[42]))
default from original: 0
default from copy:     unbound

You see that if the original hash table was created with a second argument to makeTable, then the copy is liable to return a wrong default value. To fix this problem is a bit challenging but interesting.

We need to pass a second argument to the call to makeTable within copyTable. Moreover, we need to pass the correct value for this second argument, based on the given table. Unfortunately, there is no built-in function in SKILL which will access the default value of a hash table.

Calculating the default value of a hash table

The following function returns the default value of the hash table. It is pretty efficient, and works on the following algorithm.

• Choose a token key; we use the symbol t in this case, but any key would work.
• Save away the length of the hash table, by calling (length hash).
• Save away the value of hash[t].
• Call (remove hash t) to delete the t key from the table if it is there. If t is not a hash key, no harm done.
• Evaluate hash[t] once t has been removed. That access will return the default value which we want getHashDefault to return.
• Determine whether t was in the table at the start by comparing the hash table size before and after deleting the t key.
• Before getHashDefault returns the default value, restore t to the hash table if and only if it was there originally.

To return a particular value from a SKILL function but first do some cleanup, use prog1.

To do the restoration we save the old value of hash[t] before calling remove, then check the length of the hash before and after calling remove. If the length of the hash changed, then twas a key originally, so restore the value to the saved value. Otherwise, there is no need to restore anything.

(defun getHashDefault (hash)
  (let ((old_size (length hash))
        (old_value hash[t]))
    (remove t hash)
    (prog1 hash[t]
      (unless ((length hash) == old_size)
        hash[t] = old_value))))

A working version of copyTable

All that remains to have a working version is to update copyTable to make use of getHashDefault.

(defun copyTable (hash)
  (append (makeTable (getHashPrintName hash)
                     (getHashDefault hash))
          hash))

We can test it to make sure it works.

(letseq ((table (makeTable 'data 0))
         (copy (copyTable table)))
  (printf "default from original: %L\n" table[42])
  (printf "default from copy:     %L\n" copy[42]))
default from original: 0
default from copy:     0

Summary

In this blog post we saw several things when trying to implement a robust version of copyTable.

• How to find the default value of a hash table
• How to find the print name of a hash table
• How the symbol unbound behaves differently in SKILL vs SKILL++
• How the symbols ? and ?? behave as hash keys.
• Some details of the SKILL function makeTable, in particular the first and second arguments.
• How append and remove work with hash tables.
• How to use the function tableToList to serialize the contents of a hash table.

Jim Newton