Geometry shaders can be used in two places: in instance definitions that reference a shader instead of an object, light, camera, or instance group:
instance "name"
geometry function
...
end instance
or in phenomena, where they can be attached to the geometry root:
declare phenomenon
...
geometry function
...
end declare
In either case, the geometry shader is evaluated during scene preprocessing before any other operation such as rendering starts. The geometry shader is expected to create an object, light, camera, or instance group and add the tag of the created element to the result. The result pointer passed as its first argument always has the type miTag *, and the shader must be declared as
declare shader
geometry "shader_name" ("parameter_decl")
...
end declare
If result is a null tag, the created element can be returned directly using this pointer, otherwise the geometry shader must check whether the type of the item to which result refers is miSCENE_GROUP. If it refers to an instance group, the created entities can be put into this group, otherwise the shader must create the group and put the element to which result refers together with the entities created by the geometry shader into the instance group. The newly created instance group must be returned in result in this case. This ensures that geometry shaders always return either an object or an instance group, and that they can be chained such that the end result is a group. There is a shader interface function available for adding scene entities to result which takes all the above rules into account; refer to mi_geoshader_add_result.
Creation of geometry requires an altogether different set of shader interface functions. The shader is basically doing the same thing that an object or other definition expressed in the .mi language is doing, and must have the same functionality available to it. In fact, the shader interface functions for geometry shaders closely model the .mi language features: there are begin/end functions for most top-level entities and complex sub-entities, and various helper functions to create and attach complex information. Many of these return pointers to the created data structures that let the geometry shader store primitive data directly without the use of a shader interface function.
Geometry shaders must include geoshader.h after shader.h. Note that this file is not compatible between mental ray 2.0 and mental ray 2.1 because a number of features that were introduced in mental ray 2.1 required extra fields in data structures such as miBox. This means that geometry shaders compiled for mental ray 2.0 will not, in general, work with mental ray 2.1, and vice versa!
This chapter provides an example and lists all shader interface API functions and data structure declarations available to geometry shaders. It does not explain the structure and order of specific API calls needed to create a scene. Refer to the ``Scene Description Language'' section for the sequence of operations necessary to create a scene element, and to appendix A for the correspondence between API calls and the entities in the scene description language they create.
In any case, writing geometry shaders is far more complex than writing another type of shader. Although a geometry shader has free run of the entire scene database, and is free to alter or create any part of it, it is generally a good idea to avoid doing too much with them. Instead, it is usually preferable to pass objects, materials, and shaders as input parameters instead of creating entire graphs in geometry shaders, although this is possible and sometimes needed in geometry shaders used in phenomena. Normally, geometry shaders should not modify existing parts of the scene because this can confuse future incremental changes and scene traversal during preprocessing, which geometry shaders are part of. Instead, the element to be modified should be passed to the geometry shader as a parameter of type geometry, and be used as a template to create a new, modified element.
All geometry shader API calls and data structures may only be used in geometry shaders, not any other type of shader. Conversely, no regular shader interface function related to rendering may be used in a geometry shader because it is called before rendering begins. Geometry shaders are called during the same stage as displacement shaders, but before the displacement shaders of objects created by this geometry shader.