A reference of AppStream's public API can be fount on the API reference pages (api/index.html).
AppStream provides a second library with building blocks to compose metadata catalogs.
It is used by the appstreamcli compose
utility as well as
AppStream Generator (https://github.com/ximion/appstream-generator).
You can find its API description on the libappstream-compose API reference (compose-api/index.html).
Please keep in mind that the compose library interface is still not fully stabilized.
If you want to use the PackageKit API, look at the PackageKit reference manual (https://www.freedesktop.org/software/PackageKit/gtk-doc/).