Interacting with Unreal Engine

The Arcware Pixel Streaming WebSDK provides a seamless interface for web applications to interact with Unreal Engine-powered experiences. This interaction is crucial for creating an immersive and interactive user experience, where the actions taken on the web interface can directly affect the game or application running in Unreal Engine.

---

Overview

The WebSDK facilitates communication between your web-app and the Unreal Engine Application using the WebRTC protocoll. This connection enables the transmission of user inputs from the web to the Unreal Engine, allowing for real-time interaction.

---

Sending Input to Unreal Engine

To send input to the Unreal Engine, the ArcwareApplication exposes a method called emitUIInteraction. This method allows you to send structured data that Unreal Engine can recognize and respond to as if it were native input.

/** Emit an event towards the UE Application.
* Mainly used to bubble events like a button press or other command inputs towards the UE Application. */
Application.emitUIInteraction(descriptor: object | string): void;

Example: emitUIInteraction

// Define the interaction command as per Unreal Engine's expected format
const interactionCommand = {
  type: "interact",
  payload: {
    action: "Jump",
    value: "true"
  }
};

// Send the interaction command to Unreal Engine
Application.emitUIInteraction(interactionCommand);

In the above example, Application is an instance of the ArcwareApplication class, which encapsulates the streaming and interaction logic. The emitUIInteraction method takes an object that defines the type of interaction and its payload.

Receiving Responses

The WebSDK also allows you to handle responses from Unreal Engine. When you send an interaction command, Unreal Engine can send back a response that you can capture and handle within your web application.

Handling Application Responses

To handle responses from Unreal Engine, you can use the getApplicationResponse method provided by the WebSDK. This method sets up a callback function that will be called whenever a response is received.

/** Request a response from you UE Application.
* The returned response depends on the implementation of your UE Application. */
Application.getApplicationResponse(callback: (response: string) => void): void;

Example: getApplicationResponse

// Set up a response handler
Application.getApplicationResponse((response) => {
  console.log("Received response from Unreal Engine:", response);
  // Handle the response accordingly
});

In this example, any response from Unreal Engine will be logged to the console, and you can add additional logic to handle the response as needed.

Summary

The interaction between your web application and Unreal Engine is a two-way street. You can send commands to Unreal Engine and handle the responses using the emitUIInteraction and getApplicationResponse methods, respectively. This powerful feature of the WebSDK opens up a myriad of possibilities for creating interactive and engaging web-based experiences powered by Unreal Engine.

Important Consideration for Developers

Please note that in order for these interaction features to work as expected, corresponding input handlers and response mechanisms must be properly set up on the Unreal Engine side. This typically involves scripting within the Unreal Engine project to define how it should process incoming commands and what responses should be sent back. Without this setup, the WebSDK will not be able to communicate effectively with the Unreal Engine application. For more info how to set up Unreal Engine for this purpose, click on the link below.

01.2. Pixel Streaming Input / Json messages