2012/01/14

Catching Keyboard Events in iOS


Some notes before starting ...

Things explained here are NOT possible using public APIs so this definitely violates AppStore's rules. However, if you want to take your chances and implement this for the AppStore or perhaps for Jailbroken iPhones ... keep on reading :) I think this time it is harder to get caught since this doesn't need to link against private frameworks or add private headers, etc.

The question is: How?

The short answer

The trick is in accessing GSEventKey struct memory directly and check certain bytes to know the keycode and flags of the key pressed. Below code is almost self explanatory and should be put in your UIApplication subclass.

#define GSEVENT_TYPE 2
#define GSEVENT_FLAGS 12
#define GSEVENTKEY_KEYCODE 15
#define GSEVENT_TYPE_KEYUP 11

NSString *const GSEventKeyUpNotification = @"GSEventKeyUpHackNotification";

- (void)sendEvent:(UIEvent *)event
{
    [super sendEvent:event];

    if ([event respondsToSelector:@selector(_gsEvent)]) {

        // Key events come in form of UIInternalEvents.
        // They contain a GSEvent object which contains 
        // a GSEventRecord among other things

        int *eventMem;
        eventMem = (int *)[event performSelector:@selector(_gsEvent)];
        if (eventMem) {

            // So far we got a GSEvent :)
            
            int eventType = eventMem[GSEVENT_TYPE];
            if (eventType == GSEVENT_TYPE_KEYUP) {
                
                // Now we got a GSEventKey!
                
                // Read flags from GSEvent
                int eventFlags = eventMem[GSEVENT_FLAGS];
                if (eventFlags) { 

                    // This example post notifications only when 
                    // pressed key has Shift, Ctrl, Cmd or Alt flags

                    // Read keycode from GSEventKey
                    int tmp = eventMem[GSEVENTKEY_KEYCODE];
                    UniChar *keycode = (UniChar *)&tmp;

                    // Post notification
                    NSDictionary *inf;
                    inf = [[NSDictionary alloc] initWithObjectsAndKeys:
                      [NSNumber numberWithShort:keycode[0]],
                      @"keycode",
                      [NSNumber numberWithInt:eventFlags], 
                      @"eventFlags",
                      nil];
                    [[NSNotificationCenter defaultCenter] 
                        postNotificationName:GSEventKeyUpNotification 
                                      object:nil
                                    userInfo:userInfo];
                }
            }
        }
    }
}


If you are asking yourself "Where all those #defines come from?", "Application subclass?, I don't have such a thing in code" then please read the long answer, I hope you find if helpful :)

The long answer:

UIEvents are simple wrappers of GSEventRefs, which contain a GSEventRecord struct in it. Usually we only treat UIEvents representing touch events because the UIApplication will not dispatch other events to our views or objects we create. These are UIInternalEvents and can represent accelerometer events, volume events, keyboard events, etc.

  1. In order to intercept all events our application receives we need to override sendEvent: method in our UIApplication subclass.
  2. We need to access the GSEvent and check it is a key event and then check its flags (Shift, Cmd, Ctrl, Alt). Its easy as that, the problem is we don't know how to get the type and flags of GSEvent, it is a private API :(

- GSEvent objects -
So I did the homework and according to Kenny TM in here and here, GSEvents looks like:
typedef struct __GSEvent {
    CFRuntimeBase _base;
    GSEventRecord record;
} GSEvent;
typedef struct __GSEvent* GSEventRef;
typedef struct GSEventRecord {
    GSEventType type; // 0x8 //2
    GSEventSubType subtype;    // 0xC //3
    CGPoint location;     // 0x10 //4
    CGPoint windowLocation;    // 0x18 //6
    int windowContextId;    // 0x20 //8
    uint64_t timestamp;    // 0x24, from mach_absolute_time //9
    GSWindowRef window;    // 0x2C //
    GSEventFlags flags;    // 0x30 //12
    unsigned senderPID;    // 0x34 //13
    CFIndex infoSize; // 0x38 //14
} GSEventRecord;
typedef struct GSEventKey {
 GSEvent _super;
 UniChar keycode, characterIgnoringModifier, character; // 0x38, 0x3A, 0x3C
 short characterSet;  // 0x3E
 Boolean isKeyRepeating; // 0x40
} GSEventKey;
Headers seems to be a bit old (iOS3~4) but things haven't changed so much. What is for sure is that keycode is right next to infoSize. (Isn't this the fun of private APIs?).
Everything we have to do now is to count bytes from the start of the memory of GSEvent
int *eventMem;
eventMem = (int *)[event performSelector:@selector(_gsEvent)];
GSEventType is at 2
GSEventFlags at 12 and
Unichar keycode at index 15.
I do some checks in the way but that is basically all. :)

Yes, Apple might change GSEvent at anytime so if you are a challenger and would like to submit this to the AppStore, at least do do this conditionally:
// I am lazy to do the proper check here in the post :)
[super sendEvent:event];
if ([[[UIDevice currentDevice] systemVersion] intValue] == 5) {
    ...
}

If you use it or tried to, it would be awesome you drop a line in the comments. (^-^)/
I still have not completed a sample for this little hack but in the mean you can check this this gist out where you can find some keycodes and masks.

References


6 comments :

RyanCumley said...

Thanks for posting this! I've spent the past few hours poking around and had figured out that bluetooth keyboard events were wrapped in a UIInternalEvent, but not much more than that. Your code was perfect!

I'm still looking for a legit way around the GSEvent calls, and will let you know if I come up with anything.

Thanks again!

-Ryan Cumley

Ignacio Enriquez said...

If there is a "better" way of getting GSEvents I would like to know too!
Unfortunately AFAIK, at least in iOS5 and below they are part of the private APIs :(

Jie Hou said...

Thanks for the post! I wonder does the keyboard events only apply when using an external physical keyboard? When I try this hack using virtual keyboard on touch screen, it seems only generates events "3001", which is Hand event not KeyUp nor KeyDown event.

Is there any way to catch the input of a virtual keyboard on screen?

Ignacio Enriquez said...

Hello Jie.
I haven't tried that. Regarding software keyboard events using the UITextViewDelegate has been enough so far.

My guess is that touch events are catch by the keyboard view which interacts with the input system directly and tells it exactly what character should be input without needing to create physical keyboard events.

I think the hack will need to be done in the software keyboard class and not in UIApplication.

I will investigate about this and write my findings some time latter this week :)

new said...

Thanks for this it was really useful. Unfortunately Apple did catch it and the app was rejected. I'm left wondering how the existing apps in the appstore got away with it (or if they found some other trick).

Gray Lin said...

I tried the posted code, it can get touch event, but can't get keyboard event.

iOS 7 / iPad mini / xcode5