Skip to content

Commit

Permalink
Project import generated by Copybara.
Browse files Browse the repository at this point in the history
GitOrigin-RevId: d073f8e21be2fcc0e503cb97c6695078b6b75310
  • Loading branch information
MediaPipe Team authored and chuoling committed Feb 27, 2021
1 parent 39309be commit 350fbb2
Show file tree
Hide file tree
Showing 755 changed files with 16,377 additions and 11,061 deletions.
5 changes: 4 additions & 1 deletion .bazelrc
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,12 @@ common --experimental_repo_remote_exec

# Basic build settings
build --jobs 128
build --define='absl=1'
build --define='absl=1' # for gtest
build --enable_platform_specific_config

# Enable stack traces
test --test_env="GTEST_INSTALL_FAILURE_SIGNAL_HANDLER=1"

# Linux
build:linux --cxxopt=-std=c++17
build:linux --host_cxxopt=-std=c++17
Expand Down
2 changes: 2 additions & 0 deletions MANIFEST.in
Original file line number Diff line number Diff line change
Expand Up @@ -8,3 +8,5 @@ include README.md
include requirements.txt

recursive-include mediapipe/modules *.tflite *.txt *.binarypb
exclude mediapipe/modules/objectron/object_detection_3d_chair_1stage.tflite
exclude mediapipe/modules/objectron/object_detection_3d_sneakers_1stage.tflite
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ Hair Segmentation

[]() | [Android](https://google.github.io/mediapipe/getting_started/android) | [iOS](https://google.github.io/mediapipe/getting_started/ios) | [C++](https://google.github.io/mediapipe/getting_started/cpp) | [Python](https://google.github.io/mediapipe/getting_started/python) | [JS](https://google.github.io/mediapipe/getting_started/javascript) | [Coral](https://github.com/google/mediapipe/tree/master/mediapipe/examples/coral/README.md)
:---------------------------------------------------------------------------------------- | :-------------------------------------------------------------: | :-----------------------------------------------------: | :-----------------------------------------------------: | :-----------------------------------------------------------: | :-----------------------------------------------------------: | :--------------------------------------------------------------------:
[Face Detection](https://google.github.io/mediapipe/solutions/face_detection) | ✅ | ✅ | ✅ | | | ✅
[Face Detection](https://google.github.io/mediapipe/solutions/face_detection) | ✅ | ✅ | ✅ | | ✅ | ✅
[Face Mesh](https://google.github.io/mediapipe/solutions/face_mesh) | ✅ | ✅ | ✅ | ✅ | ✅ |
[Iris](https://google.github.io/mediapipe/solutions/iris) | ✅ | ✅ | ✅ | | |
[Hands](https://google.github.io/mediapipe/solutions/hands) | ✅ | ✅ | ✅ | ✅ | ✅ |
Expand All @@ -44,7 +44,7 @@ Hair Segmentation
[Object Detection](https://google.github.io/mediapipe/solutions/object_detection) | ✅ | ✅ | ✅ | | | ✅
[Box Tracking](https://google.github.io/mediapipe/solutions/box_tracking) | ✅ | ✅ | ✅ | | |
[Instant Motion Tracking](https://google.github.io/mediapipe/solutions/instant_motion_tracking) | ✅ | | | | |
[Objectron](https://google.github.io/mediapipe/solutions/objectron) | ✅ | | | | |
[Objectron](https://google.github.io/mediapipe/solutions/objectron) | ✅ | | | | |
[KNIFT](https://google.github.io/mediapipe/solutions/knift) | ✅ | | | | |
[AutoFlip](https://google.github.io/mediapipe/solutions/autoflip) | | | ✅ | | |
[MediaSequence](https://google.github.io/mediapipe/solutions/media_sequence) | | | ✅ | | |
Expand Down
4 changes: 2 additions & 2 deletions WORKSPACE
Original file line number Diff line number Diff line change
Expand Up @@ -38,8 +38,8 @@ http_archive(

http_archive(
name = "rules_foreign_cc",
strip_prefix = "rules_foreign_cc-master",
url = "https://github.com/bazelbuild/rules_foreign_cc/archive/master.zip",
strip_prefix = "rules_foreign_cc-main",
url = "https://github.com/bazelbuild/rules_foreign_cc/archive/main.zip",
)

load("@rules_foreign_cc//:workspace_definitions.bzl", "rules_foreign_cc_dependencies")
Expand Down
44 changes: 22 additions & 22 deletions docs/framework_concepts/calculators.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,26 +67,26 @@ class CalculatorBase {

// The subclasses of CalculatorBase must implement GetContract.
// ...
static ::MediaPipe::Status GetContract(CalculatorContract* cc);
static absl::Status GetContract(CalculatorContract* cc);

// Open is called before any Process() calls, on a freshly constructed
// calculator. Subclasses may override this method to perform necessary
// setup, and possibly output Packets and/or set output streams' headers.
// ...
virtual ::MediaPipe::Status Open(CalculatorContext* cc) {
return ::MediaPipe::OkStatus();
virtual absl::Status Open(CalculatorContext* cc) {
return absl::OkStatus();
}

// Processes the incoming inputs. May call the methods on cc to access
// inputs and produce outputs.
// ...
virtual ::MediaPipe::Status Process(CalculatorContext* cc) = 0;
virtual absl::Status Process(CalculatorContext* cc) = 0;

// Is called if Open() was called and succeeded. Is called either
// immediately after processing is complete or after a graph run has ended
// (if an error occurred in the graph). ...
virtual ::MediaPipe::Status Close(CalculatorContext* cc) {
return ::MediaPipe::OkStatus();
virtual absl::Status Close(CalculatorContext* cc) {
return absl::OkStatus();
}

...
Expand Down Expand Up @@ -199,7 +199,7 @@ name and index number. In the function below input are output are identified:
// c++ Code snippet describing the SomeAudioVideoCalculator GetContract() method
class SomeAudioVideoCalculator : public CalculatorBase {
public:
static ::mediapipe::Status GetContract(CalculatorContract* cc) {
static absl::Status GetContract(CalculatorContract* cc) {
cc->Inputs().Index(0).SetAny();
// SetAny() is used to specify that whatever the type of the
// stream is, it's acceptable. This does not mean that any
Expand All @@ -209,13 +209,13 @@ class SomeAudioVideoCalculator : public CalculatorBase {
cc->Outputs().Tag("VIDEO").Set<ImageFrame>();
cc->Outputs().Get("AUDIO", 0).Set<Matrix>();
cc->Outputs().Get("AUDIO", 1).Set<Matrix>();
return ::mediapipe::OkStatus();
return absl::OkStatus();
}
```
## Processing
`Process()` called on a non-source node must return `::mediapipe::OkStatus()` to
`Process()` called on a non-source node must return `absl::OkStatus()` to
indicate that all went well, or any other status code to signal an error
If a non-source calculator returns `tool::StatusStop()`, then this signals the
Expand All @@ -224,12 +224,12 @@ input streams will be closed (and remaining Packets will propagate through the
graph).
A source node in a graph will continue to have `Process()` called on it as long
as it returns `::mediapipe::OkStatus(`). To indicate that there is no more data
to be generated return `tool::StatusStop()`. Any other status indicates an error
has occurred.
as it returns `absl::OkStatus(`). To indicate that there is no more data to be
generated return `tool::StatusStop()`. Any other status indicates an error has
occurred.
`Close()` returns `::mediapipe::OkStatus()` to indicate success. Any other
status indicates a failure.
`Close()` returns `absl::OkStatus()` to indicate success. Any other status
indicates a failure.
Here is the basic `Process()` function. It uses the `Input()` method (which can
be used only if the calculator has a single input) to request its input data. It
Expand All @@ -238,13 +238,13 @@ and does the calculations. When done it releases the pointer when adding it to
the output stream.
```c++
::util::Status MyCalculator::Process() {
absl::Status MyCalculator::Process() {
const Matrix& input = Input()->Get<Matrix>();
std::unique_ptr<Matrix> output(new Matrix(input.rows(), input.cols()));
// do your magic here....
// output->row(n) = ...
Output()->Add(output.release(), InputTimestamp());
return ::mediapipe::OkStatus();
return absl::OkStatus();
}
```

Expand Down Expand Up @@ -312,7 +312,7 @@ namespace mediapipe {
//
class PacketClonerCalculator : public CalculatorBase {
public:
static ::mediapipe::Status GetContract(CalculatorContract* cc) {
static absl::Status GetContract(CalculatorContract* cc) {
const int tick_signal_index = cc->Inputs().NumEntries() - 1;
// cc->Inputs().NumEntries() returns the number of input streams
// for the PacketClonerCalculator
Expand All @@ -322,10 +322,10 @@ class PacketClonerCalculator : public CalculatorBase {
cc->Outputs().Index(i).SetSameAs(&cc->Inputs().Index(i));
}
cc->Inputs().Index(tick_signal_index).SetAny();
return ::mediapipe::OkStatus();
return absl::OkStatus();
}

::mediapipe::Status Open(CalculatorContext* cc) final {
absl::Status Open(CalculatorContext* cc) final {
tick_signal_index_ = cc->Inputs().NumEntries() - 1;
current_.resize(tick_signal_index_);
// Pass along the header for each stream if present.
Expand All @@ -336,10 +336,10 @@ class PacketClonerCalculator : public CalculatorBase {
// the header for the input stream of index i
}
}
return ::mediapipe::OkStatus();
return absl::OkStatus();
}

::mediapipe::Status Process(CalculatorContext* cc) final {
absl::Status Process(CalculatorContext* cc) final {
// Store input signals.
for (int i = 0; i < tick_signal_index_; ++i) {
if (!cc->Inputs().Index(i).Value().IsEmpty()) {
Expand All @@ -364,7 +364,7 @@ class PacketClonerCalculator : public CalculatorBase {
}
}
}
return ::mediapipe::OkStatus();
return absl::OkStatus();
}

private:
Expand Down
14 changes: 7 additions & 7 deletions docs/framework_concepts/gpu.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,19 +66,19 @@ calculator derived from base class GlSimpleCalculator. The GPU calculator
// See GlSimpleCalculator for inputs, outputs and input side packets.
class LuminanceCalculator : public GlSimpleCalculator {
public:
::mediapipe::Status GlSetup() override;
::mediapipe::Status GlRender(const GlTexture& src,
const GlTexture& dst) override;
::mediapipe::Status GlTeardown() override;
absl::Status GlSetup() override;
absl::Status GlRender(const GlTexture& src,
const GlTexture& dst) override;
absl::Status GlTeardown() override;

private:
GLuint program_ = 0;
GLint frame_;
};
REGISTER_CALCULATOR(LuminanceCalculator);

::mediapipe::Status LuminanceCalculator::GlRender(const GlTexture& src,
const GlTexture& dst) {
absl::Status LuminanceCalculator::GlRender(const GlTexture& src,
const GlTexture& dst) {
static const GLfloat square_vertices[] = {
-1.0f, -1.0f, // bottom left
1.0f, -1.0f, // bottom right
Expand Down Expand Up @@ -128,7 +128,7 @@ REGISTER_CALCULATOR(LuminanceCalculator);
glDeleteVertexArrays(1, &vao);
glDeleteBuffers(2, vbo);

return ::mediapipe::OkStatus();
return absl::OkStatus();
}
```
Expand Down
12 changes: 6 additions & 6 deletions docs/framework_concepts/graphs.md
Original file line number Diff line number Diff line change
Expand Up @@ -219,23 +219,23 @@ packet timestamps 0, 1, 2, 3, ...
```c++
class UnitDelayCalculator : public Calculator {
public:
 static ::util::Status FillExpectations(
 static absl::Status FillExpectations(
     const CalculatorOptions& extendable_options, PacketTypeSet* inputs,
     PacketTypeSet* outputs, PacketTypeSet* input_side_packets) {
   inputs->Index(0)->Set<int>("An integer.");
   outputs->Index(0)->Set<int>("The input delayed by one time unit.");
   return ::mediapipe::OkStatus();
   return absl::OkStatus();
 }
 ::util::Status Open() final {
 absl::Status Open() final {
   Output()->Add(new int(0), Timestamp(0));
   return ::mediapipe::OkStatus();
   return absl::OkStatus();
 }
 ::util::Status Process() final {
 absl::Status Process() final {
   const Packet& packet = Input()->Value();
   Output()->AddPacket(packet.At(packet.Timestamp().NextAllowedInStream()));
   return ::mediapipe::OkStatus();
   return absl::OkStatus();
 }
};
```
Expand Down
11 changes: 5 additions & 6 deletions docs/getting_started/android_archive_library.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,8 @@ each project.
2. Run the Bazel build command to generate the AAR.
```bash
bazel build -c opt --host_crosstool_top=@bazel_tools//tools/cpp:toolchain --fat_apk_cpu=arm64-v8a,armeabi-v7a \
bazel build -c opt --host_crosstool_top=@bazel_tools//tools/cpp:toolchain \
--fat_apk_cpu=arm64-v8a,armeabi-v7a --strip=ALWAYS \
//path/to/the/aar/build/file:aar_name
```
Expand Down Expand Up @@ -86,16 +87,14 @@ each project.
Build the MediaPipe binary graph and copy the assets into
app/src/main/assets, e.g., for the face detection graph, you need to build
and copy
[the binary graph](https://github.com/google/mediapipe/blob/master/mediapipe/examples/android/src/java/com/google/mediapipe/apps/facedetectiongpu/BUILD#L41),
[the tflite model](https://github.com/google/mediapipe/tree/master/mediapipe/models/face_detection_front.tflite),
[the binary graph](https://github.com/google/mediapipe/blob/master/mediapipe/examples/android/src/java/com/google/mediapipe/apps/facedetectiongpu/BUILD#L41)
and
[the label map](https://github.com/google/mediapipe/blob/master/mediapipe/models/face_detection_front_labelmap.txt).
[the face detection tflite model](https://github.com/google/mediapipe/tree/master/mediapipe/modules/face_detection/face_detection_front.tflite).
```bash
bazel build -c opt mediapipe/mediapipe/graphs/face_detection:mobile_gpu_binary_graph
cp bazel-bin/mediapipe/graphs/face_detection/mobile_gpu.binarypb /path/to/your/app/src/main/assets/
cp mediapipe/models/face_detection_front.tflite /path/to/your/app/src/main/assets/
cp mediapipe/models/face_detection_front_labelmap.txt /path/to/your/app/src/main/assets/
cp mediapipe/modules/face_detection/face_detection_front.tflite /path/to/your/app/src/main/assets/
```
![Screenshot](../images/mobile/assets_location.png)
Expand Down
2 changes: 1 addition & 1 deletion docs/getting_started/hello_world_android.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ node: {
output_stream: "luma_video"
}
# Applies the Sobel filter to luminance images sotred in RGB format.
# Applies the Sobel filter to luminance images stored in RGB format.
node: {
calculator: "SobelEdgesCalculator"
input_stream: "luma_video"
Expand Down
2 changes: 1 addition & 1 deletion docs/getting_started/hello_world_cpp.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ nav_order: 1
`PrintHelloWorld()` function, defined in a [`CalculatorGraphConfig`] proto.
```C++
::mediapipe::Status PrintHelloWorld() {
absl::Status PrintHelloWorld() {
// Configures a simple graph, which concatenates 2 PassThroughCalculators.
CalculatorGraphConfig config = ParseTextProtoOrDie<CalculatorGraphConfig>(R"(
input_stream: "in"
Expand Down
8 changes: 6 additions & 2 deletions docs/getting_started/hello_world_ios.md
Original file line number Diff line number Diff line change
Expand Up @@ -492,6 +492,9 @@ in our app:
if (![self.mediapipeGraph startWithError:&error]) {
NSLog(@"Failed to start graph: %@", error);
}
else if (![self.mediapipeGraph waitUntilIdleWithError:&error]) {
NSLog(@"Failed to complete graph initial run: %@", error);
}
dispatch_async(_videoQueue, ^{
[_cameraSource start];
Expand All @@ -500,8 +503,9 @@ in our app:
}];
```

Note: It is important to start the graph before starting the camera, so that the
graph is ready to process frames as soon as the camera starts sending them.
Note: It is important to start the graph before starting the camera and wait
until completion, so that the graph is ready to process frames as soon as the
camera starts sending them.

Earlier, when we received frames from the camera in the `processVideoFrame`
function, we displayed them in the `_liveView` using the `_renderer`. Now, we
Expand Down
Loading

0 comments on commit 350fbb2

Please sign in to comment.