Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement mirrored video feature for createCapture (#6441) #6703

Merged
merged 5 commits into from
Feb 27, 2024
Merged
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
63 changes: 53 additions & 10 deletions src/dom/dom.js
Original file line number Diff line number Diff line change
Expand Up @@ -2135,7 +2135,11 @@ if (navigator.mediaDevices.getUserMedia === undefined) {
* W3C documentation</a> for possible properties. Different browsers support different
* properties.
*
* The second parameter, `callback`, is optional. It's a function to call once
* The 'flipped' property is an optional property which can be set to `{flipped:true}`
* to mirror the video output.If it is true then it means that video will be mirrored
* or flipped and if nothing is mentioned then by default it will be `false`.
*
* The second parameter,`callback`, is optional. It's a function to call once
* the capture is ready for use. The callback function should have one
* parameter, `stream`, that's a
* <a href="https://developer.mozilla.org/en-US/docs/Web/API/MediaStream" target="_blank">MediaStream</a> object.
Expand All @@ -2148,6 +2152,8 @@ if (navigator.mediaDevices.getUserMedia === undefined) {
* @param {String|Constant|Object} [type] type of capture, either AUDIO or VIDEO,
* or a constraints object. Both video and audio
* audio streams are captured by default.
* @param {Object} [flipped] flip the capturing video and mirror the output with `{flipped:true}`. By
* default it is false.
* @param {Function} [callback] function to call once the stream
* has loaded.
* @return {p5.Element} new <a href="#/p5.Element">p5.Element</a> object.
Expand Down Expand Up @@ -2183,6 +2189,19 @@ if (navigator.mediaDevices.getUserMedia === undefined) {
* }
* </code>
* </div>
* <div class='notest'>
* <code>
* let capture;
*
* function setup() {
* // Create the video capture with mirrored output.
* capture = createCapture(VIDEO,{ flipped:true });
* capture.size(100,100);
* describe('A video stream from the webcam with flipped or mirrored output.');
* }
*
* </code>
* </div>
*
* <div class='notest norender'>
* <code>
Expand Down Expand Up @@ -2212,7 +2231,7 @@ if (navigator.mediaDevices.getUserMedia === undefined) {
p5.prototype.createCapture = function(...args) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we update the doc comments above this to mention the flipped parameter, and maybe add an example that uses it?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will soon update the inline documentation and give a simple good-to-go example.

p5._validateParameters('createCapture', args);

// return if getUserMedia is not supported by browser
// return if getUserMedia is not supported by the browser
if (!(navigator.mediaDevices && navigator.mediaDevices.getUserMedia)) {
throw new DOMException('getUserMedia not supported in this browser');
}
Expand All @@ -2221,12 +2240,24 @@ p5.prototype.createCapture = function(...args) {
let useAudio = true;
let constraints;
let callback;
let flipped = false;

for (const arg of args) {
if (arg === p5.prototype.VIDEO) useAudio = false;
else if (arg === p5.prototype.AUDIO) useVideo = false;
else if (typeof arg === 'object') constraints = arg;
else if (typeof arg === 'function') callback = arg;
else if (typeof arg === 'object') {
// Check if the argument is an object with a 'flipped' property
if (arg.flipped !== undefined) {
flipped = arg.flipped;
} else {
constraints = arg;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right now I think this means we either can provide flipped, or we can provide constraints on the capture size. I think we want to support both at once, but just with the flipped arg removed. Maybe something like:

if (arg.flipped !== undefined) {
  flipped = arg.flipped;
}
constraints = { ...arg };
delete constraints.flipped;

}
}
else if (typeof arg === 'function') {
callback = arg;
}
}

if (!constraints) constraints = { video: useVideo, audio: useAudio };

const domElement = document.createElement('video');
Expand All @@ -2253,6 +2284,9 @@ p5.prototype.createCapture = function(...args) {
if (domElement.width) {
videoEl.width = domElement.width;
videoEl.height = domElement.height;
if (flipped) {
videoEl.elt.style.transform = 'scaleX(-1)';
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you call yourCapture.hide() and image(yourCapture, 0, 0) to draw it to the canvas, is it flipped there too? If not, we may need to update the part of the code where media elements draw to their internal canvas and draw it scaled there too (see this comment for details: #6441 (comment))

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ya that I missed

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@davepagurek I did it , now it is working with the capture.hide() and image(capture,0,0);

Screenshot 2024-01-10 232633 Screenshot 2024-01-10 232926

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This time i have changed the _ensureCanvas() so that it works fine.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, looks great!

}
} else {
videoEl.width = videoEl.elt.width = domElement.videoWidth;
videoEl.height = videoEl.elt.height = domElement.videoHeight;
Expand All @@ -2261,9 +2295,11 @@ p5.prototype.createCapture = function(...args) {

if (callback) callback(domElement.srcObject);
});
videoEl.flipped=flipped;
return videoEl;
};


/**
* Creates a new <a href="#/p5.Element">p5.Element</a> object.
*
Expand Down Expand Up @@ -4377,7 +4413,6 @@ class MediaElement extends p5.Element {
duration() {
return this.elt.duration;
}

_ensureCanvas() {
if (!this.canvas) {
this.canvas = document.createElement('canvas');
Expand All @@ -4398,18 +4433,26 @@ class MediaElement extends p5.Element {
}

this.drawingContext.clearRect(
0,
0,
this.canvas.width,
this.canvas.height
);
0, 0, this.canvas.width, this.canvas.height);

if (this.flipped === true) {
this.drawingContext.save();
this.drawingContext.scale(-1, 1);
this.drawingContext.translate(-this.canvas.width, 0);
}

this.drawingContext.drawImage(
this.elt,
0,
0,
this.canvas.width,
this.canvas.height
);

if (this.flipped === true) {
this.drawingContext.restore();
}

this.setModified(true);
this._frameOnCanvas = this._pInst.frameCount;
}
Expand Down