Fuzzy & Pixelated PDF Copy & Paste from macOS Preview

Too long, don’t want to read.

  • Symptom. Cutting and pasting sections of PDF files from macOS / OSX Preview results in fuzzy and pixelated images where you were expecting vector PDF data to be copied and pasted.
    • Correlated symptom. You will be able to get vector data if you copy and paste an entire page instead of a selected region of a page.
  • Diagnosis. There was an app hijacking the “com.adobe.pdf” uniform type identifier (UTI). This resulted in many equivalent types not being recognized as valid PDF data on the clipboard.
  • Current fix. Identify uninstall the app by looking through the UTI registration database with lsregister. To identify the app …
    •  find the line-number of the lsregister -dump that has the currently active mapping for “com.adobe.pdf”
    • /System/Library/Frameworks/CoreServices.framework/Frameworks/LaunchServices.framework/Support/\
      lsregister -dump | grep -n -A 2 "uti:           com.adobe.pdf" 
    • For me, this showed (with some wordpress induced spacing issues)
      36795: uti: com.adobe.pdf 
      36796- description: Portable Document Format (PDF)
      36797- flags: imported inactive core apple-internal trusted 
      --
      49934: uti: com.adobe.pdf
      49935- description: PDF
      49936- flags: exported active trusted # this is the problem!  
      --
      49990: uti: com.adobe.pdf
      49991- description: PDF
      49992- flags: imported inactive trusted 
      --
      74076: uti: com.adobe.pdf
      74077- description: (null)
      74078- flags: imported inactive trusted 
      --
      77680: uti: com.adobe.pdf
      77681- description: PDF Data
      77682- flags: imported inactive untrusted
    • This indicates it’s line 49934 that has the active mapping, whereas we wanted it to be on line 36795, which is the internal apple mapping with all the appropriate metadata.  So now we need to guess at the file that this implicates. Here is my current way to do this. This isn’t entirely reliable, but I think it should work in most cases.
      # This command shows the last 10 "apps" before the offending 
      # uti type definition, the last one should be the app 
      # that is causing the problem. 
      /System/Library/Frameworks/CoreServices.framework/Frameworks/LaunchServices.framework/Support/\
      lsregister -dump | head -n 36795 | grep "CFBundleExecutable = " | tail -n 10
      
       CFBundleExecutable = "Pass Viewer";
       CFBundleExecutable = "Google Chrome Helper";
       CFBundleExecutable = "Photo Library Migration Utility";
       CFBundleExecutable = GarageBand;
       CFBundleExecutable = PIPAgent;
       CFBundleExecutable = PowerChime;
       CFBundleExecutable = "Problem Reporter";
       CFBundleExecutable = gephi;
       CFBundleExecutable = rcd;
       CFBundleExecutable = Notability;
    • So the offending app is likely to be Notability.
    • I uninstalled Notability and everything resumed it’s normal behavior regarding copy & paste after I reset the UTI database.
      /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/LaunchServices.framework/Versions/A/Support/\
      lsregister -kill -seed -r
    • (Then I reinstalled Notability and everything was bad again!)


Editorial comments

I was annoyed by this because it seems like this should be impossible. The app that caused the problem, Notability, was installed via the macOS app store. These are apparently super-sandboxed in terms of what they can do on the computer. (Or so I’ve read in the past about these apps.) Yet this particular app was able to disrupt a key feature in my daily usage of the mac between two entirely separate applications!

  • How can this totally corrupt my system even outside of apps that don’t use this information?
  • This appears to be a simple app configuration mistake. Why aren’t these checked as part of the App Store application process?

The reason to write these editorial comments is not to complain, but to see something change. It’s easier and faster (and more productive as this is mostly outside my day-to-day work) not to care. But the best software and products are built by those who care deeply about all the issues and I want to work with software built by such people.

What might change?  It’d be great if some check of overriding core Apple types was added to the macOS app store review process. And I’m hoping that the Notability folks fix the issue with their app now too.


The full story

This is a tale of a super subtle bug that has been bugging me for a while! (pun intended).

A feature I love.

The starting point is one of the greatest things about using macOS. I spend substantial time editing slides that have graphs, equations, and figures. This often involves pulling an equation or figure out of a published paper to comment on it in the slides.

Back when I used to do this in Windows, it always irked me that you couldn’t get the vector data into the Powerpoint slide. Instead, you’d enlarge the region of the PDF you wanted to “snip” out of the document and then take a screenshot or use the Acrobat snip tool. This may have changed more recently, but this was still the case as of Windows 7.

Once I moved to the mac, it was lovely to be able to snip a region of a paper and get the full vector information into Powerpoint. (Yes, I still use Powerpoint on the mac, I happened to learn its features better than Keynote and haven’t had time to convert.) 

Here’s an example from a talk I gave at the Univ. of Madison last year.

example-use

The equations on this slide were copied directly from our paper on this topic. I do not want to retype these for a talk! The vector copy and paste versions look beautiful at any resolution and don’t display any pixelation.

This aspect alone made it worth moving to the mac!


And it all comes crashing down…

At some point this fall, I followed my usual framework of sniping from a PDF in Preview and then pasting into Powerpoint.

Imagine my surprise when, upon rescaling, I saw …

the-problem

Yeesh! What is that? That’s terrible! Now let’s see, what had I changed recently?

  • Upgraded to macOS Sierra
  • Upgraded to Powerpoint 2016 (which already had some rendering issues with fuzzy fonts.)

I still had Powerpoint 2011 around, which used to work! So I gave that a shot. No dice, it still had the same pixelated copy and paste behavior now.

Maybe, I thought, Sierra has changed the default “paste” time from the Vector format. But PowerPoint can paste in a variety of formats, so let’s see if the vector PDF was hiding in there…

paste-special-not-working

No luck. Only the annoying pixelated TIFF format.

So this made me conclude it was likely macOS Sierra and Powerpoint that have somehow become incompatible with vector graphics cut and paste.

So then I tried it with Keynote.

Same issue. 😦

Now I can’t imagine Apple shipping Sierra with such an obvious bug, so that’s when I figured something else must be going on…

Surely others must have similar issues? But my attempts to locate any one were futile. I did, however, find someone who had described similar behavior in an older version of OS X who came up with the following workaround. [Sorry, I lost the link as this was months ago]

  • Copying an entire page from Preview via the sidebar worked for this individual.
  • And it turns out it worked for me too! I got the nice, crisp vector graphics again!

This gave me a new, albeit annoying, workflow.

  1. Select my region.
  2. Copy it to the clipboard.
  3. Create a new preview document from the clipboard.
  4. Then copy that entire page (which was just my area)
  5. Paste into Powerpoint.

This added two steps. Not a huge deal in the grand scheme of things, but irritating.

A few months pass …

I can’t remember what it was, but at some point, I needed to do this more than once or twice — maybe 10 or 15 things.

In this case, a few extra steps become highly irritating.

So I looked into it again.

I could replicate the previous vector behavior on a machine with Powerpoint 2011 & El Capitan. And everything worked great with Keynote on the same version. So I posted a plea to twitter.

A few folks (thanks!) confirmed that everything worked nicely for them. So … hmm… it must be me.

My range of possibilities at this point

  • Some setting in the bowels of the system got flipped that turns off Preview’s vector cut & paste…
  • Something with Universal Clipboard between iOS and Sierra is causing the problem.
  • Something I’ve installed has taken over my clipboard and is pixelating my nice vector images for the sake of “compatibility”. (This sounds like something that would happen on Windows!)
  • I have a horrible virus that’s stealing everything I copy to the clipboard and is probably sending it to …

Of course, my assumption is the first item.

Descending into the inner workings …

Or the straw that broke the camel’s back.

This came up again this week as I was putting together the intro slides for my class and realized I only had them in Keynote. And then later because I wanted to copy some old PDF data into Notability to update my notes for a class. I was doing screen-captures and all sorts of other stuff that used to be common in my Windows workflows.

After another Google search turned up some info on people who were having trouble with copy and paste on Sierra. They suggested booting into Safe Mode and see if the problem occurs there. Yup! Still does 😦

On a hunch, I tried looking up how to turn off universal clipboard after I saw something I was working with transfer over to my iPad. On the second page of that discussion they mentioned “pasteboard” and some of the various types of information here.

This led me to try and understand the clipboard (or pasteboard) view of the world.

I worked with Windows for a very long time and recall their standard Clipboard viewing utility (turns out the mac has something similar in Finder, Edit-Show Clipboard). I also know there are many ways to debug the clipboard contents. I figured something like that must exist for the mac too. (All of my mac development work has been on the command line, so I’m extremely unfamiliar with XCode.)

So eventually I track down Clipboard Viewer to look at raw clipboard data.
This was invaluable!

I checked a computer that had the issue (mine) against another one where
everything seemed to be working.

Notice the difference? It’s right there in the first entry. (Note that I recreated these images on my own machine after the fact, so the PDF data changes a little bit, but the important difference is there.)

“com.adobe.pdf” is listed as a type when things work (other, the second picture) , and missing when things don’t work (mine, the first picture)

At this point, I assumed that the “dyn.ah62d…” was some sort of internal identifier that should have been mapped to “com.adobe.pdf”. Which would completely fit with the symptoms! Maybe somehow a database of these entries became corrupted and disabled this ability.

(Note that you get a very different set of types if you copy an entire page from Preview, so that would explain why this operation would still work.)

At this point, I thought it  was just a matter of tracking down what that database.

Google to the rescue! (Eventually…)

Google. Apple PDF source pasteboard
Find. https://discussions.apple.com/thread/6625785?start=0&tstart=0
Insight. Hey, someone else has had this problem too! But they didn’t solve it either, but it seems to exactly fit my description!

Google. pasteboard osx dyn
Find. http://stackoverflow.com/questions/31833291/what-clipboard-type-class-strings-does-os-x-javascript-for-applications-recognis
Insight. hey, these things are called Uniform Type Identifiers.

Then I found more info from Apple on these Uniform Type Identifiers.

Still thinking that it’s a corrupted database, I continued on…

Google. Apple’s Uniform Type Identifier database corrupted
Find. http://apple.stackexchange.com/questions/47319/how-can-i-make-os-x-recognize-new-file-extensions
Insight. Oh, this has something to do with “Launch Services” and “lsregister

I ran

/System/Library/Frameworks/CoreServices.framework/Frameworks/LaunchServices.framework/Support/\
lsregister -dump

to see what it showed. Oof, that’s a lot of info! Tons of stuff for “com.adobe.pdf” too.

But this also said that applications can register new stuff.

This is the full string for that type.

Google. “dyn.ah62d4rv4gu8yc6durvwwaycek2uha2pxsvw0e55bsmwca7d3sbwu”
Find. absolutely no results! Ack!

Google. “com.adobe.pdf” uti corrupted
Find. nothing much

Then I read more in the apple docs and found out about “Dynamic UTIs”.

Google. “dynamic uti”
Find. https://alastairs-place.net/blog/2012/06/06/utis-are-better-than-you-think-and-heres-why/
Insight. Ugh, this is unlikely to be corrupted as that’s just a mapping to another type.

So let’s decode that dyn.ah62d4rv4gu8yc6durvwwaycek2uha2pxsvw0e55bsmwca7d3sbwu string, which is is just a base-32 encoded type. (Which is insanely clever, actually, but that’s besides the point.)

Incidentally, I wish there was a little online decoder for these apple base32 type strings. I had to write a program to convert to a standard base-32 encoding, and then feed it in.

If you rewrite this as a standard base-32 string (instead of an apple encoded one) it is:
h4yd2nr2gq6uc4dqnrssauceiyqhayltorswe33bojsca5dz
which decodes to
?0=6:4=Apple PDF pasteboard type

Here’s my Julia code to do this.

mapstr="abcdefghkmnpqrstuvwxyz0123456789"
X = Dict{Char,Int}()
for i=1:length(mapstr)
  X[mapstr[i]]=i-1
end
mapstr2 = "abcdefghijklmnopqrstuvwxyz234567"
Y = Dict{Int,Char}()
for i=1:length(mapstr2)
  Y[i-1] = mapstr2[i]
end
# padded with additional a's to make decoding work
x = "h62d4rv4gu8yc6durvwwaycek2uha2pxsvw0e55bsmwca7d3sbwuaaaa"
map( x -> Y[X[x]], x)

So this told me, somehow, Apple PDF pasteboard type is not being mapped to “com.adobe.pdf”.

Back in that massive lsregister -dump result, now I had something specific to look for beyond “com.adobe.pdf”, which shows up everywhere.

In fact, “Apple PDF” shows up exactly once.

type    id:            7356
        uti:           com.adobe.pdf
        description:   Portable Document Format (PDF)
        flags:         imported inactive core apple-internal trusted 
        icon: 
        conforms to:   public.data, public.composite-content
        tags:          .pdf, 'PDF ', application/pdf, "Apple PDF pasteboard type"

At this point, it was time for bed.


The next morning, I looked at the database on a computer that was working.

type    id:            7356
        uti:           com.adobe.pdf
        description:   Portable Document Format (PDF)
        flags:         imported active core apple-internal trusted 
        icon: 
        conforms to:   public.data, public.composite-content
        tags:          .pdf, 'PDF ', application/pdf, "Apple PDF pasteboard type"

It said “active”! (And note, that I just changed “inactive” to “active” and in reality, the type id changed too…)

Finally, I had a clue that seemed like it was on the right track.

Now, at this point, I actually have no clue what “inactive” means in the flags.

I still think I’m looking to correct a corrupted database.

Google. lsregister “inactive”
Find. http://stackoverflow.com/questions/18984025/mac-file-association-not-working-why-or-how-to-debug and https://sayzlim.net/clean-your-mac-weekly-routine/
Insight. Hey, maybe I can reset launch services!
Find. http://mjtsai.com/blog/2013/04/23/rebuilding-the-launch-services-database/
Insight. Seems like this is really a thing.
Find. http://furbo.org/2013/04/22/launch-services-woes/
Insight. Okay, worth a shot.

And I try

/System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/LaunchServices.framework/Versions/A/Support/\
lsregister -kill -seed -r

But it doesn’t fix the problem and I still see the flags: inactive 😦


At this point, I realized it was likely an app corrupting the world instead of a mis-configured database.

So I wanted to look at all Info.plist files that declare these types to see what was mentioning “Apple PDF” (which is where I thought the problem was)

find /Applications -name Info.plist -ipath \
  "/Applications/*.app/Contents/Info.plist" -depth 4 \
  -exec grep "Apple PDF" "{}" ";"

(via the page http://www.jaharmi.com/2008/11/22/find_info_plist_files_for_applications_on_mac_os_x)

This showed nothing. So there are no types registering to “Apple PDF” anything!

I think I was going through other tabs or other hits on the google searches because the next link in my history is: https://lists.apple.com/archives/carbon-dev/2013/Jan/msg00009.html

Which is where I learned about CoreTypes and the system Info.plist.

I then found the Default Apple Entry for “com.adobe.pdf”. Which I was hoping would have an “inactive” flag I could change.

No such luck.

But that got me thinking …

What was active for “com.adobe.pdf” then?

Back to grep…

/System/Library/Frameworks/CoreServices.framework/Frameworks/LaunchServices.framework/Support/\
lsregister -dump | grep -C 10 "uti:           com.adobe.pdf"

notability-pdf

Ahah. Notability!

The other app I installed around that same point in time in order to look at notes on the new iPad Pro!

Uninstalling — didn’t fix it.

But uninstalling and resetting the database

/System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/LaunchServices.framework/Versions/A/Support/\
lsregister -kill -seed -r

Shows that the Apple Uniform Type Information entry was happily active again.

Copy and pasting now worked exactly as it should!

  • Clipboard Viewer shows “com.adobe.pdf” as expected
  • Powerpoint and Keynote paste PDF files
  • Paste special in PowerPoint shows the right thing.

And the example I started with worked perfectly.

pasting-working

Mission accomplished.

Postscript.

Much of this work-through and analysis is probably uninformed to those who live and breathe with these issues and I suspect these individuals could have sussed out the solution much faster.

But I have a working system again — and can hopefully help others who have the same problem. And I’m extremely grateful to all those who posted their insights that I could find via Google.

Advertisements
This entry was posted in Uncategorized. Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s