{ "cells": [ { "cell_type": "markdown", "id": "a6000001", "metadata": {}, "source": [ "# Interaction and Selection\n", "\n", "The WebGPU scene provides built-in camera controls (rotate, pan, zoom) automatically.\n", "This notebook shows how to add **selection** — running callbacks when the user clicks\n", "on a rendered object.\n", "\n", "> **Note:** Selection requires a live Python kernel and only works in interactive\n", "> Jupyter sessions (Pyodide) or in ngapp. Exported HTML retains camera interaction\n", "> but does not support selection callbacks." ] }, { "cell_type": "markdown", "id": "a6000002", "metadata": {}, "source": [ "## Basic selection\n", "\n", "To enable selection:\n", "\n", "1. Register a click handler on the input handler that calls `scene.select(x, y)`.\n", "2. Register callbacks via `renderer.on_select(callback)` — the scene identifies which\n", " renderer owns the clicked pixel and dispatches to its callback.\n", "\n", "The callback receives a `SelectEvent` with:\n", "\n", "| Property | Description |\n", "|----------|-------------|\n", "| `event.uint32[0]` | Instance index (which shape was clicked) |\n", "| `event.float32[1]` | Scalar value at that point |\n", "| `event.calculate_position(scene.options)` | 3D world position |" ] }, { "cell_type": "code", "execution_count": null, "id": "a6000003", "metadata": {}, "outputs": [], "source": [ "import webgpu.jupyter as wj\n", "import numpy as np\n", "from webgpu.shapes import ShapeRenderer, generate_cylinder, generate_cone\n", "from webgpu import Colormap, Labels, Colorbar\n", "\n", "rand = np.random.random\n", "N = 10\n", "thickness = 0.1\n", "length = 0.3\n", "\n", "cylinder = generate_cylinder(8, thickness)\n", "cone = generate_cone(8, thickness)\n", "\n", "cmap_cone = Colormap(10, 11, \"viridis\")\n", "cmap_cyl = Colormap(0, 1)\n", "\n", "cylinders = ShapeRenderer(\n", " cylinder, rand((N, 3)), length * rand((N, 3)), rand(2 * N), colormap=cmap_cyl\n", ")\n", "cones = ShapeRenderer(\n", " cone, rand((N, 3)), length * rand((N, 3)), 10.0 + rand(2 * N), colormap=cmap_cone\n", ")\n", "\n", "label = Labels([\"Click an object\"], [[-1, -1]], font_size=16)\n", "scene = wj.Draw([cylinders, cones, label, Colorbar(cmap_cone), Colorbar(cmap_cyl, (-0.9, 0.75))])\n", "\n", "\n", "def set_label(s):\n", " label.labels[0] = s\n", " label.set_needs_update()\n", " scene.render()\n", "\n", "\n", "def on_select_cone(ev):\n", " set_label(f\"Selected cone {ev.uint32[0]} with value {ev.float32[1]:.3f}\")\n", "\n", "\n", "def on_select_cyl(ev):\n", " set_label(f\"Selected cylinder {ev.uint32[0]} with value {ev.float32[1]:.3f}\")\n", "\n", "\n", "cones.on_select(on_select_cone)\n", "cylinders.on_select(on_select_cyl)\n", "\n", "\n", "def on_click(ev):\n", " if ev[\"button\"] == 0:\n", " scene.select(ev[\"canvasX\"], ev[\"canvasY\"])\n", "\n", "\n", "scene.input_handler.on_click(on_click)" ] }, { "cell_type": "markdown", "id": "a6000004", "metadata": {}, "source": [ "## How selection works under the hood\n", "\n", "Each renderer that supports selection has **two** rendering pipelines:\n", "\n", "- The **rendering pipeline** draws the object normally.\n", "- The **selection pipeline** renders object metadata into a `vec4` selection\n", " texture (specified by `select_entry_point`).\n", "\n", "The default `ShapeRenderer` selection shader writes:\n", "\n", "| Channel | Content |\n", "|---------|---------| \n", "| `x` | `@RENDER_OBJECT_ID@` — auto-substituted renderer ID |\n", "| `y` | `bitcast(input.position.z)` — depth for unprojection |\n", "| `z` | `input.instance` — which instance |\n", "| `w` | `bitcast(input.color.x)` — scalar value |\n", "\n", "When `scene.select(x, y)` is called it reads the selection texture at that pixel,\n", "identifies the renderer from channel 0, reads depth from channel 1, and dispatches the `SelectEvent` with\n", "channels 2–3 as user data." ] }, { "cell_type": "markdown", "id": "a6000005", "metadata": {}, "source": [ "## Custom selection shader\n", "\n", "Override `select_entry_point` to store different data in the two user-data\n", "channels. The example below stores the surface normal (x and y components)\n", "instead of instance + value. This lets you detect whether the side or a cap\n", "of a cylinder was clicked. The 3D click position is still available for free\n", "via `calculate_position` (which uses the depth in channel 1)." ] }, { "cell_type": "code", "execution_count": null, "id": "a6000006", "metadata": {}, "outputs": [], "source": [ "select_shader = \"\"\"\n", "@fragment fn my_select_shader(\n", " input: ShapeVertexOut,\n", ") -> @location(0) vec4 {\n", " return vec4(\n", " @RENDER_OBJECT_ID@,\n", " bitcast(input.position.z),\n", " bitcast(input.normal.x),\n", " bitcast(input.normal.y),\n", " );\n", "}\n", "\"\"\"\n", "\n", "\n", "class MyShapeRenderer(ShapeRenderer):\n", " select_entry_point = \"my_select_shader\"\n", "\n", " def get_shader_code(self):\n", " return super().get_shader_code() + select_shader\n", "\n", "\n", "mycylinders = MyShapeRenderer(cylinder, rand((N, 3)), rand((N, 3)), rand(2 * N))\n", "mylabel = Labels([\"\"], [[-1, -0.8]], font_size=14)\n", "\n", "\n", "def on_select_mycyl(ev):\n", " nx, ny = ev.float32[0], ev.float32[1]\n", " nz = max(0, 1 - nx**2 - ny**2) ** 0.5\n", " pos = ev.calculate_position(myscene.options)\n", " mylabel.labels[0] = (\n", " f\"normal=({nx:.2f},{ny:.2f},{nz:.2f}) \"\n", " f\"pos=({pos[0]:.2f},{pos[1]:.2f},{pos[2]:.2f})\"\n", " )\n", " mylabel.set_needs_update()\n", " myscene.render()\n", "\n", "\n", "mycylinders.on_select(on_select_mycyl)\n", "\n", "myscene = wj.Draw([mycylinders, mylabel])\n", "myscene.input_handler.on_click(lambda ev: myscene.select(ev[\"canvasX\"], ev[\"canvasY\"]))" ] }, { "cell_type": "markdown", "id": "a6000007", "metadata": {}, "source": [ "## Disabling selection\n", "\n", "Renderers that should not participate in selection (labels, gizmos, colorbars)\n", "set `select_entry_point = \"\"`. This skips the selection pipeline entirely for\n", "that renderer." ] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "name": "python", "version": "3.13.11" } }, "nbformat": 4, "nbformat_minor": 5 }