Support multi-touch behaviour with panning and scrolling

[matis-dk]

Description

I’m trying to implement something similar to the gesture behaviour on the default list view on iOS. Eg. the “Reminders” app inside iOS.

It consist of a scrollable view with some list item's inside. When you long-press a list item, it lets you rearrange where the item belong. At the surface, this implementation seems quite simple - a ScrollView wrapping around multiple ListItem component's that is using the Gesture.Pan().activateAfterLongPress(250).

The code could look something like this

import { useRef } from "react";
import { Dimensions, View } from "react-native";
import {
  Gesture,
  GestureDetector,
  ScrollView,
} from "react-native-gesture-handler";
import Animated, {
  useAnimatedStyle,
  useSharedValue,
  withSpring,
} from "react-native-reanimated";
import { Text } from "../restyled/Text";

const items = [
  { id: "id_John", name: "John" },
  { id: "id_Jane", name: "Jane" },
  { id: "id_Jack", name: "Jack" },
  { id: "id_Jill", name: "Jill" },
  { id: "id_Joe", name: "Joe" },
  { id: "id_Jim", name: "Jim" },
  { id: "id_Joy", name: "Joy" },
];

const listItemHeight = 75;
const listItemWidth = Dimensions.get("window").width;

type Position = { x: number; y: number };

export function List() {
  return (
    <ScrollView style={{ flex: 1, gap: 12 }}>
        {items.map((item, index) => (
          <ListItem key={item.id} id={item.id} />
        ))}
    </ScrollView>
  );
}

type ListItemProps = {
  id: string;
};

function ListItem(props: ListItemProps) {
  const { id } = props;

  const sharedSelectedItem = useSharedValue<string | null>(null);
  const sharedSelectedPosStart = useRef(
    useSharedValue<Position>({
      x: 0,
      y: 0,
    })
  ).current;
  const sharedSelectedPos = useRef(
    useSharedValue<Position>({
      x: 0,
      y: 0,
    })
  ).current;

  const panGesture = Gesture.Pan()
    .activateAfterLongPress(250)
    .onStart((e) => {
      sharedSelectedItem.value = id;
      sharedSelectedPosStart.value = { x: e.absoluteX, y: e.absoluteY };
    })
    .onUpdate((e) => {
      sharedSelectedPos.value = {
        y: e.absoluteY - sharedSelectedPosStart.value.y,
        x: e.absoluteX - sharedSelectedPosStart.value.x,
      };
    })
    .onFinalize((e) => {
      sharedSelectedItem.value = null;
      sharedSelectedPos.value = {
        x: withSpring(0),
        y: withSpring(0),
      };
    });

  const animatedStyle = useAnimatedStyle(() => {
    const active = sharedSelectedItem.value === id;
    return {
      backgroundColor: active ? "blue" : "gray",
      opacity: active ? 1 : 0.5,
      transform: [
        {
          translateX: sharedSelectedPos.value.x,
        },
        {
          translateY: sharedSelectedPos.value.y,
        },
      ],
    };
  });

  return (
    <GestureDetector gesture={panGesture}>
      <Animated.View
        key={props.id}
        style={[
          {
            height: listItemHeight,
            width: listItemWidth,
            justifyContent: "center",
          },
          animatedStyle,
        ]}
      >
        <Text style={{ textAlign: "center" }}>{props.id}</Text>
      </Animated.View>
    </GestureDetector>
  );
}

The problem arise when the scrollable list is expanding beyond the device height dimensions. Because if you would like to drag a ListItem to the bottom of the list, then you need to be able to support scrolling while panning (dragging the item around). To support this, iOS implement 2 different scroll behaviors.

1. Hot areas in top and bottom of the screen, that imperatively scroll at a constant pace.
2. Support for scroll with a secondary finger, while the panning is active with the primary finger.

I think that bullet 1. is possible by keeping track of the fingers absolute position on the screen, and activate scroll imperatively, when the primary finger is entering hot areas in the top or bottom of the screen.

But bullet 2. doesn't seem to be possible by the primitive provider by RNGH currently.
By default, ScrollView's seems to be dismissed when a pan-gesture is active. The simultaneousHandlers prop allow the ScrollView gesture, and Gesture.Pan, to be active simultaneously - but this let the primary finger, pan and scroll at the same time.

What seems to be the crux of the problem, is that when a Gesture.Pan is active, RNGH is disabling ScrollView gestures globally, and not locally to the current gesture in action. If theGesture.Pan instead recognized and ignored the scrollview events, for the local pan-gesture in action, it would allow the secondary finger to scroll. What seems to be missing is a way to distingquish between disabling and ignoring ScrollView events for a given gesture.

The multi-touch behavior combining pan with the primary finger, and scrolling with secondary finger, is used multiple places inside iOS. E.g when.

• rearranging widgets on widget screen (vertical scroll)
• rearranging icons on one of the homescreen (horizontal scroll)
• rearranging items on a native list view (vertical scroll)

I noticed that Gesture.Native is able to wrap and listen to events on the ScrollView, but I'm unable to figure out how this can be composed with behavior im describing above.

Steps to reproduce

The code could look something like this

import { useRef } from "react";
import { Dimensions, View } from "react-native";
import {
  Gesture,
  GestureDetector,
  ScrollView,
} from "react-native-gesture-handler";
import Animated, {
  useAnimatedStyle,
  useSharedValue,
  withSpring,
} from "react-native-reanimated";
import { Text } from "../restyled/Text";

const items = [
  { id: "id_John", name: "John" },
  { id: "id_Jane", name: "Jane" },
  { id: "id_Jack", name: "Jack" },
  { id: "id_Jill", name: "Jill" },
  { id: "id_Joe", name: "Joe" },
  { id: "id_Jim", name: "Jim" },
  { id: "id_Joy", name: "Joy" },
];

const listItemHeight = 75;
const listItemWidth = Dimensions.get("window").width;

type Position = { x: number; y: number };

export function List() {
  return (
    <ScrollView contentContainerStyle={{ flex: 1 }}>
      <View style={{ gap: 12 }}>
        {items.map((item, index) => (
          <ListItem key={item.id} id={item.id} />
        ))}
      </View>
    </ScrollView>
  );
}

type ListItemProps = {
  id: string;
};

function ListItem(props: ListItemProps) {
  const { id } = props;

  const sharedSelectedItem = useSharedValue<string | null>(null);
  const sharedSelectedPosStart = useRef(
    useSharedValue<Position>({
      x: 0,
      y: 0,
    })
  ).current;
  const sharedSelectedPos = useRef(
    useSharedValue<Position>({
      x: 0,
      y: 0,
    })
  ).current;

  const panGesture = Gesture.Pan()
    .activateAfterLongPress(250)
    .onStart((e) => {
      sharedSelectedItem.value = id;
      sharedSelectedPosStart.value = { x: e.absoluteX, y: e.absoluteY };
    })
    .onUpdate((e) => {
      sharedSelectedPos.value = {
        y: e.absoluteY - sharedSelectedPosStart.value.y,
        x: e.absoluteX - sharedSelectedPosStart.value.x,
      };
    })
    .onFinalize((e) => {
      sharedSelectedItem.value = null;
      sharedSelectedPos.value = {
        x: withSpring(0),
        y: withSpring(0),
      };
    });

  const animatedStyle = useAnimatedStyle(() => {
    const active = sharedSelectedItem.value === id;
    return {
      backgroundColor: active ? "blue" : "gray",
      opacity: active ? 1 : 0.5,
      transform: [
        {
          translateX: sharedSelectedPos.value.x,
        },
        {
          translateY: sharedSelectedPos.value.y,
        },
      ],
    };
  });

  return (
    <GestureDetector gesture={panGesture}>
      <Animated.View
        key={props.id}
        style={[
          {
            height: listItemHeight,
            width: listItemWidth,
            justifyContent: "center",
          },
          animatedStyle,
        ]}
      >
        <Text style={{ textAlign: "center" }}>{props.id}</Text>
      </Animated.View>
    </GestureDetector>
  );
}

Details

Snack or a link to a repository

https://snack.expo.dev/@matis/99d863

Gesture Handler version

2.12.0

React Native version

0.72.4

Platforms

Android, iOS

JavaScript runtime

None

Workflow

None

Architecture

None

Build type

None

Device

None

Device model

No response

Acknowledgements

Yes

[matis-dk]

Demo showcasing scroll and panning simultaneously.

Dragging a list item from iOS's "Reminders" app:
https://github.com/software-mansion/react-native-gesture-handler/assets/32526593/d42a6357-e316-4241-8cfd-4b6e2a65ee51

Dragging a iOS widget between the different homescreen:
https://github.com/software-mansion/react-native-gesture-handler/assets/32526593/3cffb182-9b4c-414c-9e61-af83db2db570

[matis-dk]

@j-piasecki can you by any chance confirm that this is in fact currently not possible with the primitives provided by RNGH?

[zebriot]

Here is a snippet i used with to handle simulatneously multiple gestures with native elements, the key is to have the native element as direct child to your gesture detector

You can do something like this,

  const nativeGesture = Gesture.Native();
  const composedGestures = Gesture.Simultaneous(myPanGesture, nativeGesture);
  const ScrollComponent = (props: any) => {
    return (
     <GestureDetector gesture={composedGestures}>
      <ScrollView {...props} />
      </GestureDetector>
    );
  };

  return (
        <Flashlist
          renderScrollComponent={ScrollComponent}
            {...otherProps}
        />
  );

This will handle both gestures simultaneously, I am just using translationX in my myPanGesture gesture so i havent tried yet with others.

[latekvo]

Hi @matis-dk

The end result you described is as far as i know only possible to achieve on Android phones for now.

The code snippet sent by @zebriot is a good example of how you can achieve these results, the key is to wrap your scroll component with Gesture.Native().

Here's another example which works well on Android:

Collapsed code

import React from 'react';
import { StyleSheet, Text, View, ScrollView } from 'react-native';
import { Gesture, GestureDetector } from 'react-native-gesture-handler';
import Animated, {
  useAnimatedStyle,
  useSharedValue,
  withSpring,
} from 'react-native-reanimated';

export default function EmptyExample() {
  const panActive = useSharedValue(false);
  const position = useSharedValue(0);
  const panPreviousPosition = useSharedValue(0);
  const scrollPreviousPosition = useSharedValue(-1);

  const scroll = Gesture.Native()
    .onTouchesMove((event) => {
      ('worklet');
      if (!panActive.value) {
        return;
      }

      const scrollPosition = event.allTouches[0].absoluteY;

      if (scrollPreviousPosition.value === -1) {
        scrollPreviousPosition.value = scrollPosition;
        return;
      }

      const delta = scrollPosition - scrollPreviousPosition.value;
      scrollPreviousPosition.value = scrollPosition;

      position.value = position.value - delta;

      // console.log('delta', delta);
      // console.log('position', position.value);
    })
    .onFinalize(() => {
      scrollPreviousPosition.value = -1;
    });

  const pan = Gesture.Pan()
    .activateAfterLongPress(250)
    .onStart(() => {
      panActive.value = true;
      position.value = 0;
    })
    .onUpdate((e) => {
      const delta = e.translationY - panPreviousPosition.value;

      position.value += delta;

      panPreviousPosition.value = e.translationY;
    })
    .onFinalize(() => {
      position.value = withSpring(0);
      panActive.value = false;
      panPreviousPosition.value = 0;
      scrollPreviousPosition.value = -1;
    });

  const elementPadding = 21;
  const elementFiller = (
    <>
      {new Array(elementPadding).fill(1).map(() => (
        <View style={styles.box} key={Math.random()}>
          <Text>Hello World!</Text>
        </View>
      ))}
    </>
  );

  const animation = useAnimatedStyle(() => ({
    ...styles.highlight,
    transform: [
      {
        translateY: position.value ?? 0,
      },
    ],
  }));

  return (
    <GestureDetector gesture={scroll}>
      <ScrollView style={styles.container}>
        {elementFiller}
        <GestureDetector gesture={pan}>
          <Animated.View style={animation}>
            <Text>Hello World!</Text>
          </Animated.View>
        </GestureDetector>
        {elementFiller}
      </ScrollView>
    </GestureDetector>
  );
}

const styles = StyleSheet.create({
  container: {
    flex: 1,
    backgroundColor: '#F5FCFF',
  },
  box: {
    padding: 40,
    backgroundColor: 'navy',
    justifyContent: 'center',
  },
  highlight: {
    padding: 40,
    backgroundColor: 'tomato',
    color: 'white',
    zIndex: 99,
  },
});

But please keep in mind, as this feature works only on Android.

---

update

as of the following PR: #3095, the Gesture.Pan() will require .simultaneousWithExternalGesture(scroll) to be added to it.

[matis-dk]

@latekvo Ok, thanks for confirming that this isn't currently possible on iOS.

@zebriot's code snippet runs both gestures (the scroll and pan) simultaneously for the same finger/pointer - this isn't the behaviour i'm looking for.

In the iOS interactions I'm referring to, one finger is solely responsible for panning/dragging the item, and the secondary finger for scrolling.

[matis-dk]

WWDC 2014 Session 235 - Advanced Scroll Views and Touch Handling Techniques seems very relevant.

The problem described at 33:35 inside the presentation is somewhat similar our current issue.
The solution proposed in the video is basically to disable, and re-enable the scrollview's pan gesture recognizer for the touch in action. Here is a touch demo implemented in Swift. This works as you would expect.

When trying to use the setNativeProps on the scrollview's ref inside RN, the scrolling seems to be disabled entirely for all touches.