Skip to main content

2024.05.1b

Huge update once again, this time, focusing more on the user frontend experience and heavily refactoring the backend for developers to extend support for their devices/software. Finger tracking, new animator, Facial Tracking API additions, desktop gestures, better gestures, VideoPlayer API, invite requests, and so much more!

Changes​

  • Added Finger Tracking
    • All Avatars now have their fingers tracked from the current IFingerCurler interface.
    • Developers, see below for more information on IFingerCurler
  • Desktop Crouching
    • Hit C to crouch
  • Desktop Crawling
    • Hit X to crawl
  • New default Animation/Animators
    • This new Animator includes the following animations
      • Walk
        • Walk Forward
        • Walk Backward
        • Walk Left
        • Walk Right
      • Run
        • Run Forward
        • Run Backward
        • Run Left
        • Run Right
      • Crouch
        • Crouch Walk
        • Crouch Walk Left
        • Crouch Walk Right
      • Lay Down
        • Crawl
      • Jump
      • Falling
        • Falling Idle
      • Left Hand Finger Curling
      • Right Hand Finger Curling
  • Desktop Finger Tracking
    • Hold Left Shift for the Left Hand
    • Hold Right Shift for the Right Hand
    • While holding a Shift, hit a number from 1-9 to activate a Gesture in order of its properties, as listed below:
      • Fist (1), OpenHand (2), Point (3), Peace (4), OkHand (5), RockAndRoll (6), Gun (7), ThumbsUp (8)
      • The current GestureIdentifier does not change the numbers hit, but will change what number the gesture returns
  • Added Invite Requests
    • If your friend is in a world and not on DoNotDisturb status, you will see a Request button on their profile. Upon selecting the Request button, an Invite Request will be sent to the user.
    • The user you sent the invite request to can decide to accept the invite request
  • Added Badge Implementation with BadgeRankHandler
    • See API changes for more information
  • Added UnityVideoPlayer as an integration of IVideoPlayer
    • This will be expanded to add support for other video players in the future
  • Added IGestureIdentifier for creating custom Gesture numbers
    • By default, these are the current implementations
      • HypernexGesture
      • CGesture
      • VGesture
    • You can change which GestureIdentifier you are using in the User tab in Settings
    • See API changes for more information
  • Added IFingerCurler for creating custom Finger Curlers
    • By default, there are three implementations
      • DesktopFingerCurler.Left
        • For Desktop, Left Hand
      • DesktopFingerCurler.Right
        • For Desktop, Right Hand
      • HandGetter
        • For VR
    • See API changes for more information
  • Added ICustomFaceExpression for creating custom Facial Expressions from VRCFaceTracking data
    • For example, there are three implementations
      • CombinedEyeLid
      • TongueX
      • TongueY
    • See API changes for more information
  • Smoothened networked weights
    • This makes weight changed feel smoother, without ruining the quality of the value
  • Added SmoothFloat for smoothening a float value
    • See API changes for more information
  • Reworked FingerCalibration for the new system of Finger Curling.
  • Added ability for AnimatorCreator.SetParameter<T>(string, T, CustomPlayableAnimator, bool, bool) to set parameters in the MainAnimator
  • Created RotationOffsetDriver to handle offset rotations for AvatarCreators.
    • See API changes for more information
  • All HandleCameras are now destroyed upon leaving or joining an instance
    • This is a temporary addition until the planned Cameras menu is added
  • Fixed a bug where FullBody-Tracking would not work
  • Fixed a bug where FullBody Trackers would not be registered as IsTracking when they were tracking
    • This fix is mostly a hack, so if you see a tracker dot at center not tracking, just wait a couple of seconds before it disappears.
  • Fixed bug where a NetPlayer's hips would not move or rotate when using Full-Body Tracking
  • Fixed bug where you could not create more than one HandleCamera

API Changes​

BadgeRankHandler​

The new BadgeRankHandler allows publishers to edit nameplates to handle Ranks for Badges. Here is an example of a verified badge being added to users with the verified rank.

caution

BadgeRankHandler MUST have a public, parameter-less constructor, otherwise the base class will throw an exception!

using System.Linq;
using Hypernex.UI.Templates;
using HypernexSharp.APIObjects;
using TMPro;
using UnityEngine;
using UnityEngine.UI;

public class VerifiedRank : BadgeRankHandler
{
    // This is for ranks only
    public override bool IsRank => true;
    // These are the ranks this will Apply to
    public override Rank[] TargetRanks => new[]{Rank.Verified, Rank.Moderator, Rank.Admin, Rank.Owner};

    public override void ApplyToNameplate(NameplateTemplate nameplateTemplate, User targetUser)
    {
        // We store any assets in the Init.BadgeRankAssets list of Objects, then just cast it to the type it is
        Texture2D verifiedTexture = (Texture2D) Init.Instance.BadgeRankAssets.ElementAt(2);
        // Working off of the status image, because it already has positioning and sizing done
        RectTransform dup = Object.Instantiate(nameplateTemplate.Status).GetComponent<RectTransform>();
        // Reparent if it isn't
        dup.SetParent(nameplateTemplate.Status.transform.parent, false);
        // Set its name
        dup.gameObject.name = "ModeratorIcon";
        // Move it up to the right-corner of the pfp
        dup.anchoredPosition3D = new Vector3(dup.anchoredPosition3D.x, dup.anchoredPosition3D.y + 82f, 0);
        // Get the image component
        Image img = dup.gameObject.GetComponent<Image>();
        // Reset the color, since the Status icon changes colors
        img.color = Color.white;
        // Create a sprite from the texture, and set the image to that sprite
        img.sprite = Sprite.Create(verifiedTexture, new Rect(0f, 0f, verifiedTexture.width, verifiedTexture.height),
            Vector2.zero);
    }
}

Here is an example that will add an icon if you have a specific badge and are verified rank.

using System.Linq;
using Hypernex.UI.Templates;
using HypernexSharp.APIObjects;
using TMPro;
using UnityEngine;
using UnityEngine.UI;

public class VerifiedRankLeafBadge : BadgeRankHandler
{
    // While this requires a rank, we want this at false since its dependent on a badge
    // You will also want your rank handler (if present) to check and see if you have this badge
    // If you do have the badge, make sure ApplyToNameplate in the rank handler does not run
    public override bool IsRank => false;
    // Name of the badge (caps sensitive)
    public override string Name => "leaf";

    private readonly Color verifiedLeafColor = new(70f/255f, 180f/255f, 48f/255f);

    public override void ApplyToNameplate(NameplateTemplate nameplateTemplate, User targetUser)
    {
        // Don't run if the user is not Verified
        if(targetUser.Rank < Rank.Verified) return;
        // Get the texture from the Init.BadgeRankAssets field
        Texture2D verifiedTexture = (Texture2D) Init.Instance.BadgeRankAssets.ElementAt(3);
        // Duplicate the status icon
        RectTransform dup = Object.Instantiate(nameplateTemplate.Status).GetComponent<RectTransform>();
        dup.SetParent(nameplateTemplate.Status.transform.parent, false);
        dup.gameObject.name = "VerifiedLeafIcon";
        // Position the icon to the top right of the pfp
        dup.anchoredPosition3D = new Vector3(dup.anchoredPosition3D.x, dup.anchoredPosition3D.y + 82f, 0);
        Image img = dup.gameObject.GetComponent<Image>();
        img.color = Color.white;
        img.sprite = Sprite.Create(verifiedTexture, new Rect(0f, 0f, verifiedTexture.width, verifiedTexture.height),
            Vector2.zero);
        // Set the color of the username text to the color defined above
        nameplateTemplate.Username.color = verifiedLeafColor;
    }
}

IVideoPlayer​

A new interface used to define support for VideoPlayer backends. Currently, this is only used to implement the already supported Unity VideoPlayer, but the vision is to add support for Virtual's FFmpeg.Unity and possibly vlc-unity (if I can ever figure out how to build it 😭)

You can see an example implementation of the UnityVideoPlayer here

IGestureIdentifier​

This interface was designed to be easily serializable and instantiable by developers (especially for plugins). It allows you to drive custom integer values for the GestureLeft and GestureRight avatar parameters. Here is an example implementation of Hypernex's gestures.

public class HypernexGesture : IGestureIdentifier
{
    public string Name => "Hypernex";
    public int Unknown => 0;
    public int Fist => 1;
    public int OpenHand => 2;
    public int Point => 3;
    public int Peace => 4;
    public int OkHand => 5;
    public int RockAndRoll => 6;
    public int Gun => 7;
    public int ThumbsUp => 8;
}

The extra Name property is used to identify the GestureIdentifier for Config values. Be sure to not have conflicting names!

IFingerCurler​

This interface was specifically designed for developers to implement their own hardware via. plugins. It allows developers to drive the finger curl values for each hand. The hand is identified with the Hand property as either Left or Right. All of the float properties are values from 0 to 1 describing how much the finger is curled, where 0 is not curled and 1 is completely curled; however, if you need the values to describe something different, override the IsCurled method. Here is a bare-bones implementation of the interface:

public class MyRandomFingerCurler : IFingerCurler
{
    public Hand Hand { get; private set; }
    private Random random;

    public MyRandomFingerCurler(bool isRight = false)
    {
        Hand = isRight ? Hand.Right : Hand.Left;
        random = new Random();
    }

    public MyRandomFingerCurler(int seed, bool isRight = false)
    {
        Hand = isRight ? Hand.Right : Hand.Left;
        random = new Random(seed);
    }

    private float RandomFloat() => (float) random.NextDouble();

    public float ThumbCurl => RandomFloat();
    public float IndexCurl => RandomFloat();
    public float MiddleCurl => RandomFloat();
    public float RingCurl => RandomFloat();
    public float PinkyCurl => RandomFloat();
}

This class will register a random finger curler, so each time the Curl value is accessed, a random value between 0 and 1 will be returned.

ICustomFaceExpression​

This interface is once again designed for developers to implement custom facial expressions with VRCFaceTracking's UnifiedTrackingData. Eventually, it will be what implements all of v2 parameters, since I can't figure out how IParameters work without OSC and because it allows greater control for parameters.

Here is an example implementation of combining eye lids:

public class CombinedEyeLid : ICustomFaceExpression
{
    public string Name => "CombinedEyeLid";

    public float GetWeight(UnifiedTrackingData data) => (data.Eye.Left.Openness + data.Eye.Right.Openness) / 2;
}

SmoothFloat​

SmoothFloat is a new tool that allows float values to be smoothly moved from value A to value B. The example below demonstrates how to use a SmoothFloat for a weight.

danger

SmoothFloats are NOT thread-safe. Do not touch them outside of the main thread.

public class MySmoothWeightHandler
{
    private SmoothFloat smoothFloat = new SmoothFloat();

    public MySmoothWeightHandler(Action<float> onSmoothWeight) => onSmoothWeight += OnWeightValue;

    // This will be smoothened
    public void GetValue() => smoothFloat.Value;

    void OnWeightValue(float value) => smoothFloat.Value = value;
}

RotationOffsetDriver​

The RotationOffsetDriver is a tool that will allow transforms to be rotated regardless of the bone's tracking space. This is useful for avatars that were ported from other engines that do not follow Unity's tracking space.

You can see how they are implemented for Head tracking on avatars here