3D world space UI

[FormalSnake]

Describe the project you are working on

A space game, and i want users to be able to input a code in a 3D space control panel

Describe the problem or limitation you are having in your project

I could implement it myself in my project, but it would be a good feature to have in the actual engine

Describe the feature / enhancement and how it helps to overcome the problem or limitation

3D space UI, like in Unity you can set UI to screen space to world space.

Describe how your proposal will work, with code, pseudo-code, mock-ups, and/or diagrams

I have this script, altho it doesnt seem to work, but it can be adjusted

extends Spatial


var mesh_size
onready var viewport = $Viewport
onready var panel_face = $MeshInstance
onready var touch_area = $MeshInstance/Area

export var display_y_scale = 0.5

var is_mouse_held = false
var is_mouse_inside = false
var last_mouse_position3D = null
var last_mouse_position2D = null

func _ready():
	touch_area.connect("mouse_entered", self, "_mouse_entered_touch_area")
	set_panel_size(display_y_scale)
	mesh_size = panel_face.mesh.size

func _mouse_entered_touch_area():
	is_mouse_inside = true

func _input(event):
	var is_mouse_event = false
	
	for mouse_event in [InputEventMouseButton, InputEventMouseMotion]:
		if event is mouse_event:
			is_mouse_event = true
	if is_mouse_event and (is_mouse_inside or is_mouse_held):
		handle_mouse(event)
	elif not is_mouse_event:
		viewport.input(event)

func handle_mouse(event):
	if event is InputEventMouseButton:
		is_mouse_held = event.pressed
	
	var mouse_pos3D = find_mouse(event.global_position)
	
	is_mouse_inside = mouse_pos3D != null
	
	if is_mouse_inside:
		mouse_pos3D = touch_area.global_transform.affine_inverse() * mouse_pos3D
	else:
		mouse_pos3D = last_mouse_position3D
		if mouse_pos3D == null:
			mouse_pos3D = Vector3.ZERO
	
	var mouse_pos2D = Vector2(mouse_pos3D.x, -mouse_pos3D.y)
	
	mouse_pos2D.x += mesh_size.x / 2
	mouse_pos2D.y += mesh_size.y / 2
	
	mouse_pos2D.x = mouse_pos2D.x / mesh_size.x
	mouse_pos2D.y = mouse_pos2D.y / mesh_size.y
	
	mouse_pos2D.x = mouse_pos2D.x * viewport.size.x
	mouse_pos2D.y = mouse_pos2D.y * viewport.size.y
	
	event.position = mouse_pos2D
	event.global_position = mouse_pos2D
	
	if event is InputEventMouseMotion:
		if last_mouse_position2D == null:
			event.relative = Vector2.ZERO
		else:
			event.relative = mouse_pos2D - last_mouse_position2D
	last_mouse_position2D = mouse_pos2D
	
	viewport.input(event)

func find_mouse(global_position):
	var camera = get_viewport().get_camera()
	
	var ray_length = 1
	var from = camera.project_ray_origin(global_position)
	var to = from + camera.project_ray_normal(global_position) * ray_length
	
	var result = get_world().direct_space_state.intersect_ray(from, to, [], touch_area.collision_layer, false, true)
	
	if result.size() > 0:
		return result.position
	else:
		 return null

func set_panel_size(y_scale):
	var touch_area_collision_shape = $MeshInstance/Area/CollisionShape
	var viewport_width = viewport.size.x
	var viewport_height = viewport.size.y
	var x_scale = 0
	var difference = viewport_width / (viewport_width + viewport_height)
	
	x_scale = y_scale / difference
	
	panel_face.mesh.size.x = x_scale
	panel_face.mesh.size.y = y_scale
	
	touch_area_collision_shape.shape.extents = Vector3(x_scale/2, y_scale/2, 0.05)

If this enhancement will not be used often, can it be worked around with a few lines of script?

Im sure people will use this a lot

Is there a reason why this should be core and not an add-on in the asset library?

So that is isn't extremely tedious

[YuriSizov]

Converted to a discussion because there is no concrete proposal for how this should work.

Keep in mind that in Godot 2D is actual 2D, it cannot be toggled to be in 3D space, which is why you need a viewport and a plane to put a texture on. While there is interest in having a way to proxy all UI drawing to 3D with less hassle, like we did with text and Label3D, it's not a trivial task to solve and needs a thought-out solution.

[Calinou]

Check the GUI in 3D demo 🙂

[eon-s]

This is something I've been asking for for a long time, some kind of 3D container that helps on building modern, diegetic UI systems #4093