Skip to content

Lo/failsafe for utterance end#1

Open
Moodbot11 wants to merge 3 commits into
mainfrom
lo/failsafe-for-utterance-end
Open

Lo/failsafe for utterance end#1
Moodbot11 wants to merge 3 commits into
mainfrom
lo/failsafe-for-utterance-end

Conversation

@Moodbot11
Copy link
Copy Markdown
Owner

@Moodbot11 Moodbot11 commented Aug 29, 2024

Summary by Sourcery

Implement a failsafe for handling incomplete speech final events in the conversation component, ensuring that the current utterance is appended if it remains unchanged for a specified duration.

Enhancements:

  • Add a failsafe mechanism to handle incomplete speech final events by appending the current utterance to the conversation if it remains unchanged for more than 1500 milliseconds.

@netlify
Copy link
Copy Markdown

netlify Bot commented Aug 29, 2024

Deploy Preview for browser-openai-bot ready!

Name Link
🔨 Latest commit e22fba2
🔍 Latest deploy log https://app.netlify.com/sites/browser-openai-bot/deploys/66d0813caf3c750008ee60b1
😎 Deploy Preview https://deploy-preview-1--browser-openai-bot.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

@sourcery-ai
Copy link
Copy Markdown

sourcery-ai Bot commented Aug 29, 2024

Reviewer's Guide by Sourcery

This pull request implements a failsafe mechanism for handling incomplete utterances in a conversation component. It modifies the handling of transcription events, introduces a new state variable for tracking the last utterance time, and adds a failsafe timer to send incomplete utterances to the LLM after a certain period of inactivity.

File-Level Changes

Change Details Files
Improved handling of empty transcripts
  • Modified condition to only add empty transcript part if speech is final
  • Changed currentUtterance state to be of type string
undefined
  • Added check to prevent processing entirely empty utterances
  • Implemented failsafe mechanism for incomplete utterances
    • Added new state variable lastUtterance to track the timestamp of the last received word
    • Implemented useEffect hook to check for incomplete utterances
    • Added logic to send incomplete utterances to LLM after 1500ms of inactivity
    • Implemented interval to regularly check for incomplete utterances
    app/components/Conversation.tsx
    Refactored utterance processing logic
    • Moved utterance display logic into a separate section
    • Added condition to update lastUtterance only when new text is received
    • Modified utterance clearing logic to set currentUtterance to undefined instead of an empty string
    app/components/Conversation.tsx

    Tips
    • Trigger a new Sourcery review by commenting @sourcery-ai review on the pull request.
    • Continue your discussion with Sourcery by replying directly to review comments.
    • You can change your review settings at any time by accessing your dashboard:
      • Enable or disable the Sourcery-generated pull request summary or reviewer's guide;
      • Change the review language;
    • You can always contact us if you have any questions or feedback.

    Copy link
    Copy Markdown

    @sourcery-ai sourcery-ai Bot left a comment

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    Hey @Moodbot11 - I've reviewed your changes - here's some feedback:

    Overall Comments:

    • Consider defining the 1500ms timeout as a named constant for better readability and maintainability.
    • Remove or comment out console.log statements before merging to production.
    Here's what I looked at during the review
    • 🟢 General issues: all looks good
    • 🟢 Security: all looks good
    • 🟢 Testing: all looks good
    • 🟡 Complexity: 1 issue found
    • 🟢 Documentation: all looks good

    Sourcery is free for open source - if you like our reviews please consider sharing them ✨
    Help me be more useful! Please click 👍 or 👎 on each comment to tell me if it was helpful.

    });
    }, [transcriptParts]);

    const [lastUtterance, setLastUtterance] = useState<number>();
    Copy link
    Copy Markdown

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    issue (complexity): Consider simplifying the incomplete speech handling logic to reduce complexity.

    The current implementation adds unnecessary complexity to handle incomplete speech scenarios. Instead of introducing new state variables and effects, consider simplifying the approach:

    1. Remove the lastUtterance state and the separate effect for the incomplete speech failsafe.
    2. Modify the existing useEffect that handles transcripts to include a simple timeout check:
    useEffect(() => {
      const parts = getCurrentUtterance();
      const content = parts
        .map(({ text }) => text)
        .join(" ")
        .trim();
    
      if (!content) return;
    
      setCurrentUtterance(content);
    
      let timeoutId;
    
      if (last && last.speech_final) {
        sendUtterance(content);
      } else {
        // Set a timeout for incomplete speech
        timeoutId = setTimeout(() => {
          sendUtterance(content);
        }, 1500);
      }
    
      return () => {
        if (timeoutId) clearTimeout(timeoutId);
      };
    }, [getCurrentUtterance, last]);
    
    const sendUtterance = (content) => {
      append({
        role: "user",
        content,
      });
      clearTranscriptParts();
      setCurrentUtterance(undefined);
    };

    This approach simplifies the logic by:

    • Eliminating the need for a separate state variable and effect
    • Using a single timeout that's cleared if speech_final is received
    • Consolidating the utterance sending logic into a single function

    This maintains the desired functionality of handling incomplete speech while reducing overall complexity and improving readability.

    * if the entire utterance is empty, don't go any further
    * for example, many many many empty transcription responses
    */
    if (!content) return;
    Copy link
    Copy Markdown

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    suggestion (code-quality): Use block braces for ifs, whiles, etc. (use-braces)

    Suggested change
    if (!content) return;
    if (!content) {


    ExplanationIt is recommended to always use braces and create explicit statement blocks.

    Using the allowed syntax to just write a single statement can lead to very confusing
    situations, especially where subsequently a developer might add another statement
    while forgetting to add the braces (meaning that this wouldn't be included in the condition).

    * incomplete speech final failsafe
    */
    useEffect(() => {
    if (!lastUtterance || !currentUtterance) return;
    Copy link
    Copy Markdown

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    suggestion (code-quality): Use block braces for ifs, whiles, etc. (use-braces)

    Suggested change
    if (!lastUtterance || !currentUtterance) return;
    if (!lastUtterance || !currentUtterance) {


    ExplanationIt is recommended to always use braces and create explicit statement blocks.

    Using the allowed syntax to just write a single statement can lead to very confusing
    situations, especially where subsequently a developer might add another statement
    while forgetting to add the braces (meaning that this wouldn't be included in the condition).

    @netlify
    Copy link
    Copy Markdown

    netlify Bot commented Aug 29, 2024

    A new user left a comment. This user must be approved by a Netlify team owner before comments can be displayed.

    Approve this user

    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Labels

    None yet

    Projects

    None yet

    Development

    Successfully merging this pull request may close these issues.

    2 participants