Cyrus' New Completely Useless Blog

Ugh... more R General

(pieced together from some random twitter comments I made)

So... R 3.6 has this new staged install mechanism that doesn't like my use of rprojroot and its makefixfile. I can get around the problem by installing with --no-staged-install but this makes me think I'm just sweeping the problem under the rug.

This makes me think maybe I should redo the way my packages load/process/provide data in the first place. I'm not a huge fan of the R package installation mechanism, but I get that it's there and that I should use it. How do folks deal with loading non-R data from packages?

One thing I see from @hadleywickham, e.g., is the use of the data-raw directory and some hand-run scripts that generate the .Rdata files. I guess this can work, but seems like a bit of a hack to me.

There are a couple of distinct issues here. One is finding data at runtime. It seems like rprojroot hasn't been updated in a while and isn't designed to deal with staged installs. The "here" package that @JennyBryan raved about a while back seems to have disappeared.

The second is the time at which very pieces of code are run (in this case to do the processing of the "external" data and the generation of .Rdata files and/or various exported variables with "canned/processed" data in them.

Anyone have good suggestions for managing both of these issues in the face of staged installs?

So far the cleanest thing I've got is to setup global variables, use .onLoad to populate them, and to, e.g., from my vignettes, use requireNamespace(...) to load the namespace and (implicitly) call my .onLoad function.

SBCL 1.5.3 released Lisp

The Release Announcement can be found here.

Some of the new goodies (from the ChangeLog) are:

  • platform support:
    • RISC-V: numerous bug fixes and improvements
    • all platforms: better run-program performance when used from multiple threads.
  • enhancement: (declaim (optimize (debug 2))) ensures compilation of top-level forms, providing better debugging for simple forms that are otherwise "byte-code interpreted" when compiled into FASLs.
  • bug fix: use of finalizers could in rare circumstances cause a crash in the garbage collector.
  • bug fix: show extended function designators, e.g. (setf foo), in the disassembler
  • optimization: reduced overhead of calling NTH/NTHCDR.
  • optimization: improved FLOAT-SIGN on DOUBLE-FLOATs on 64-bit platforms

More excellent work from, particularly, Stas Boukarev and Douglas Katzman to continue cleaning up the guts of SBCL and to, once again, trim the core size by another megabyte or two.

Fun with R scoping Lisp

R's default variable scoping behavior is weird, at least coming from the Common Lisp world.

I want the equivalent of:

(defparameter *foo* "moose")

(defun doit ()
  (print *foo*))


(let ((*foo* "bogus"))

foo is a dynamically scoped (special) variable in Common Lisp parlance. I can rebind foo in the let form, set its value to something and call a function that expects to see foo and have the called function see the new value of foo while but only while executing code in the dynamic scope of the LET form. Simplifying things a bit, if we call doit from code that is not called (directly or indirectly) from inside the LET form, doit will see the value of foo as "moose", whereas any code called from inside the LET form (or from code that calls it inside the LET block) the value of foo will be "bogus". This may seem confusing at first but becomes quite natural over time and is how I think about variable binding and scope.

R has its own ideas.

foo  <- "moose"

doit  <- function() {


local ({
    foo  <- "bogus"

This gives:

    foo  <- "bogus"

Which is obviously not what I want. Fortunately, R gives you lots of rope. There are probably many ways to achieve dynamic scoping in R, but what I've found is:

doit.dynamic  <- function() {
    dput(evalq(foo, parent.frame()))


local ({
    foo  <- "bogus"

And running that gives

+     foo  <- "bogus"
+     doit.dynamic()
+ })

Is this bad R juju?

R dput General

I just learned about dput today. This is one of the many things I've been looking for in R. Note to self: use dput to print things "readably" (in the lisp sense).

Cleaning up after upgrading quicklisp Lisp

After a new quicklisp release comes out, I usually spend a few minutes manually cleaning up ~/quicklisp/dists/quicklisp/software/ by deleting newly out of date source directories.

Turns out there's a (much) easier way:

(map nil #'ql-dist:clean (ql-dist:all-dists))

Does the job for me. Nice. Should have found this a long time ago.

Making new Java classes with ABCL Lisp

Armed Bear Common Lisp (ABCL) is an implementation of Common Lisp that runs on top of the JVM. One nice feature of this is that ABCL supports interoperability with java code so one can use java libraries from Common Lisp code.

One neat feature of ABCL is that you can create java classes directly in Common Lisp, in addition to just defining functions, lisp (CLOS) classes, generic functions, methods, etc...

Let's look at a toy example of creating a new java class:

;; let's define a new class
(defparameter *forty-two-class*
   :interfaces (list "java.lang.Comparable")
   :methods `(("compareTo" ,"java.lang.Integer" (,"java.lang.Integer")
               (lambda (this x)
                 (declare (ignore this))
                 (cond ((> 42 x) -1)
                       ((= 42 x) 0)
                       ((< 42 x) 1)
                       (t (error "bogus!"))))
               :modifiers (:public)))
   :access-flags '(:public :static :final)))

Now, from lisp, we can make a new instance of this java class as follows:

;; make an instance of it
(defparameter *forty-two* (java:jnew *forty-two-class*))

and we can call a method on it like so:

CL-USER> (java:jcall "compareTo" *forty-two* 43)

A trivial example, but I'll show some real world-examples of how this is useful in an upcoming post.

More bionic beaver trials and tribulations General

So... it's safe to say that the upgrade to bionic beaver (Ubuntu 18.04) hasn't been the smoothest. On my main development box (madbox), it took my many days to sort through the problems I mentioned the other day, but those are at least temporarily solved by using lightdm instead of gdm3. Anyway, my latest issues are 1) no video in firefox when running jackd at 48k and 2) on a somewhat related note, qjackctl won't start.

So issue 2 is only somewhat related as it doesn't really have anything to do with firefox, but I was hoping to run qjackctl to get a sense of what's going on with the various jack ports. So, try qjacktcl and get:

sly@madbox:~$ qjackctl

This application failed to start because it could not find or load the Qt platform plugin "xcb" in "".

Available platform plugins are: dxcb, eglfs, linuxfb, minimal, minimalegl, offscreen, vnc, xcb.

Reinstalling the application may fix this problem.
Aborted (core dumped)

Awesome. So it turns out that I'm not the only who has had this problem and searching for xcb failures turns up a whole mess of problems and possible solutions, none of which seemed to do anything helpful.

Finally, I came across the following Stack Overflow page which helpfully describes the QTDEBUGPLUGINS=1 environment variable.

Starting qjacktl with this set gives:

Got keys from plugin meta data ("xcb")
QFactoryLoader::QFactoryLoader() checking directory path "/usr/bin/platforms" ...
Cannot load library /usr/lib/x86_64-linux-gnu/qt5/plugins/platforms/ ( cannot open shared object file: No such file or directory)
QLibraryPrivate::loadPlugin failed on "/usr/lib/x86_64-linux-gnu/qt5/plugins/platforms/" : "Cannot load library /usr/lib/x86_64-linux-gnu/qt5/plugins/platforms/ ( cannot open shared object file: No such file or directory)"

After which doing:

sudo apt-get install --reinstall libxcb-xinerama0

fixes the problem. Hooray for debugging environment variables. Still, would have been nice if the application had told me that's where the problem lay in the first place.

Bionic Beaver/NVIDIA Workaround General

So... it turns out that a temporary solution (at least) to my NVIDIA woes is to use lightdm instead of gdm3 as the display manager.

I made the necessary changes by hand (roughly), but this article suggests that the "proper" way to do this is with dpkg-reconfigure lightdm. Perhaps I'll try dpkg-reconfigure gdm3 once bug 1768906 is fixed.

Ubuntu 18.04/NVIDIA Woes General

Well, two of three updates to Bionic Beaver (Ubuntu 18.04) went smoothly, but the third has turned into a giant disaster. On this computer the OS boots when logging in via the gdm-session-manager (?) screen, once you log in, the display completely freezes up. Have tried numerous drivers (different versions (current, beta, old, etc...), different methods of installing the drivers (NVIDIA proprietary installer, ubuntu packages, etc...), multiple graphics cards (an old GTX 770 and a brand new GTX 1070 Ti), different grub settings, etc..., all to no avail.

There's a bug here for those interested.


Hey R (#rstats), WTF? General

So I regularly find myself thinking "what where the designers of R thinking?". For today's example, consider the following contrived example. Say I've got the following function definition:

foo <- function(x) {
    cat("Hey, I've got a", x, "\n")

So far so good. I can do:

> foo("moose")
Hey, I've got a moose 

Ok, but now I want to default the argument, but I don't want to just say x = "moose" as I might want to use the same string elsewhere, only write it once, etcd... So I do:

x <- moose

foo <- function(x=x) {
    cat("Hey, I've got a", x, "\n")

and if I do: > foo("gazelle") Hey, I've got a gazelle

It looks like everything is OK. Until I go to do:

> foo()
Error in cat("Hey, I've got a", x, "\n") : 
  promise already under evaluation: recursive default argument reference or earlier problems?

WTF? Of course I want the value of x I defined above and of course setting the value of x to the currently-being-defined value makes n sense. Who designed the scoping rules here?

1 2 3 4 5 6 7 Next