at-spi stack overview (?source)

Requests from the screen reader

The Orca screen reader can explicitly request for information for a given widget of a given application, for instance getText:

  • Orca calls pyatspi's getText method on a pyatspi object corresponding to the widget
  • pyatspi's getText method (implemented in pyatspi/pyatspi/text.py) calls Atspi.Text.get_text, which is a python binding for the C atspi_text_get_text function from libatspi.so provided by at-spi2-core
  • atspi_text_get_text (implemented in at-spi2-core/atspi/atspi-text.c) creates an RPC message for the dbus sender of the application and the path for the widget, using the org.a11y.atspi.Text dbus interface, and the GetText method.

dbus transmits the message to the application.

In the case of a GTK2/3 application:

  • impl_GetText (implemented in at-spi2-atk/atk-adaptor/adaptors/text-adaptor.c) gets called by dbus, it parses the parameters (offsets of beginning and end), and calls atk_text_get_text
  • atk_text_get_text (implemented in atk1.0/./atk/atktext.c) calls the get_text method of the accessible object corresponding to the widget, for instance a textcell.
  • gail_text_cell_get_text (implemented in the gail module of gtk, gtk+2.0/modules/other/gail/gailtextcell.c or gtk_text_cell_accessible_get_text in gtk+3.0/gtk/a11y/gtktextcellaccessible.c) fetches the text from the widget corresponding to the accessible object and returns it
  • atk_text_get_text returns it
  • impl_GetText stuffes the string into a dbus response

TODO: In the case of a GTK4 application

In the case of a Java application:

  • impl_GetText and atk_text_get_text get called like for a GTK application, the get_text method is jaw_text_get_text
  • jaw_text_get_text (implemented in java-atk-wrapper/jni/src/jawtext.c) calls the get_text method of the AtkText java class
  • get_text (implemented in java-atk-wrapper/wrapper/org/GNOME/Accessibility/AtkText.java) uses the getAtIndex method of the AccessibleExtendedText interface of the widget and returns the text.
  • atk_text_get_text returns it
  • impl_GetText stuffes the string into a dbus response

In the case of mozilla:

  • impl_GetText and atk_text_get_text get called like for a GTK application, the get_text method is getTextCB
  • getTextCB (implemented in mozilla/accessible/atk/nsMaiInterfaceText.cpp) uses the TextSubstring of the HyperTextAccessible class.

In the case of a Qt4 application:

  • AtSpiAdaptor::textInterface (implemented in qt-at-spi/src/atspiadaptor.cpp) gets called by qt dbus, it parses the parameters, and calls the text method of the textInterface of the QAccessibleInterface class.
  • QAccessibleTextWidget::text (implemented in qt4-x11/src/plugins/accessible/widgets/qaccessiblewidgets.cpp) fetches the text and returns it.
  • AtSpiAdaptor::textInterface stuffes the string into a dbus response.

In the case of a Qt5 application:

  • AtSpiAdaptor::textInterface (implemented in qtbase/src/platformsupport/linuxaccessibility/atspiadaptor.cpp`) gets called by qt dbus, the rest is happening as with Qt4.

dbus transmits the response to the application.

  • atspi_text_get_text gets the string from the response and returns it

Notifications from the application

The application can notify about e.g. an insertion of text in a widget. The Orca screen reader must first register for getting the event, before the toolkit will only send events if there are listeners for them:

  • The default getListeners method (implemented in orca/src/orca/scripts/default.py) returns to call onTextInserted on object:text-changed:insert signal
  • registerScriptListeners (implemented in orca/src/orca/event_manager.py) calls _registerListener to record a callback to _enqueue on that signal
  • _registerListener calls registry.registerEventListener for this.
  • registerEventListener (implemented in python-atspi/pyatspi/registry.py) calls Atspi.EventListener.register, which is a Python binding for the C atspi_event_listener_register function.
  • atspi_event_listener_register (implemented in at-spi2-core/atspi/atspi-event-listener.c) uses dbus_bus_add_match on "org.a11y.atspi.Event" to make dbus make the _enqueue callback when the event is received. It also makes an RPC to the at-spi-registryd (conventional dbus name "org.a11y.atspi.Registry"), on the "/org/a11y/atspi/registry" path, the "org.a11y.atspi.Registry" interface, RegisterEvent method, to tell the registry to start notifying Orca about these events.

dbus transmits the RPC to at-spi2-registryd.

  • impl_register_event (implemented in at-spi2-core/registryd/registry.c) parses the request, records the registration in the registry->events list, and sends a EventListenerRegistered signal on the bus

In the case of a GTK2/3 application (quite unsure about the details):

  • When receiving the EventListenerRegistered signal in signal_filter (implemented in at-spi2-atk/atk-adaptor/bridge.c), handle_event_listener_registered is called
  • handle_event_listener_registered adds the event listening registration to the events list.

  • The Atk adaptor has added in spi_atk_register_event_listeners (implemented in at-spi2-atk/atk-adaptor/event.c) text_changed_event_listener as listener for signals "Gtk:AtkText:text-insert" and "Gtk:AtkText:text-changed"

  • Atk has added in atk_text_base_init (implemented in atk1.0/atk/atktext.c) the "text-insert" signal.
  • Gail has added gail_text_cell_update_cache as update_cache callback

  • TODO textcell to text_insert signal

  • gail_text_cell_update_cache (implemented in gtk+2.0/modules/other/gail/gailtextcell.c or gtk_text_cell_accessible_update_cache in gtk+3.0/gtk/a11y/gtktextcellaccessible.c) emits the "text_changed::insert" glib signal.
  • text_changed_event_listener (implemented in at-spi2-atk/atk-adaptor/event.c) calls emit_event
  • emit_event checks signal_is_needed before emitting the text-changed:insert event on the bus.
  • signal_is_needed checks whether the signal is in the events list.

TODO: In the case of a GTK4 application

TODO: In the case of a Qt application

TODO: In the case of a Java application

  • jaw_impl_class_init sets jaw_impl_initialize as initializer for atk class

  • when receiving events about a yet-unseen object, jaw_impl_get_instance calls atk_object_initialize

  • atk_object_initialize calls the atk class initializer, jaw_impl_initialize
  • jaw_impl_initialize calls org.GNOME.Accessibility.AtkWrapper::registerPropertyChangeListener
  • registerPropertyChangeListener calls AccessibleContext::addPropertyChangeListener to register propertyChangeListener to beans

  • AtkWrapper.java's propertyChangeListener's propertyChange get called, calls org.GNOME.Accessibility.AtkWrapper::dispatchFocusEvent.

  • dispatchFocusEvent calls focusNotify (implemented in C function Java_org_GNOME_Accessibility_AtkWrapper_focusNotify)
  • Java_org_GNOME_Accessibility_AtkWrapper_focusNotify uses jni_main_idle_add or gdk_threads_add_idle to record having to call focus_notify_handler later.
  • The glib loop calls focus_notify_handler.
  • focus_notify_handler calls atk_object_notify_state_change
  • atk_object_notify_state_change emits the signal
  • The rest is like gtk

TODO: In the case of mozilla

dbus transmits the event on the bus.

  • In Orca, the dbus filter calls _enqueue
  • _enqueue (implemented in orca/src/orca/event_manager.py) calls _addToQueue
  • _addToQueue queues an event
  • glib idle loop calls _dequeue.
  • _dequeue dequeues an event and calls _processObjectEvent
  • _processObjectEvent calls processObjectEvent
  • processObjectEvent (implemented in orca/src/orca/script.py) calls the listener for object:text-changed:insert.
  • onTextInserted (implemented in orca/src/orca/scripts/default.py for instance) updates the user output.

Startup

In the case of a GTK application:

TODO

In the case of a java application:

The javax.accessibility.assistive_technologies system property, or assistive_technologies file property contains the name of the class to load. It is loaded by loadAssistiveTechnologies, called from getDefaultToolkit in src/jdk/src/share/classes/java/awt/Toolkit.java

In the case of a mozilla application:

TODO

In the case of a QT4 application:

TODO

In the case of a QT5 application:

TODO