ProductPromotion
Logo

Elixir

made by https://0x3d.site

GitHub - kbrw/adap: Create a data stream across your information systems to query, augment and transform data according to Elixir matching rules.
Create a data stream across your information systems to query, augment and transform data according to Elixir matching rules. - kbrw/adap
Visit Site

GitHub - kbrw/adap: Create a data stream across your information systems to query, augment and transform data according to Elixir matching rules.

GitHub - kbrw/adap: Create a data stream across your information systems to query, augment and transform data according to Elixir matching rules.

ADAP

Awesome (big) Data Augmentation Pipeline

Create a data stream across your information systems to query, augment and transform data according to Elixir matching rules.

Build Status

See the generated documentation for more detailed explanations.

The principle is:

  • to make each element hop from node to node in order to be processed using the locally present data.
  • that any node at any time can emit new elements in the pipeline stream
  • to construct processing units on each node on demand. They can die at any time to free memory or because of an exception: they will be restarted on demand.
  • to pull elements by chunk in order to allow long processing time without the need of any back-pressure mechanism.

Let's see a processing pipe example:

  • the input is a product stream : stream of {:product,%{field1: value1, field2: value2}}
  • user@jsonserver1 contains a json file "/color.json" containing a COLOR mapping
  • user@jsonserver2 contains a json file "/size.json" containing a SIZE mapping
  • you want to map product color and size according to these mappings
  • you want to add a field "deleted" when the mapped color is red
Adap.Piper.defpipe ColorPipe, [{ColorPipe.Rules,[]}]
defmodule JSONMap do
  use Adap.Unit.Simple, ttl: 1_000
  def init(mapping), do: 
    {:ok,File.read!("/#{mapping}.json") |> JSON.decode!}
  def node("color"), do: :"user@jsonserver1"
  def node("size"), do: :"user@jsonserver2"
end
defmodule ColorPipe.Rules do
  use Adap.Piper, for: :product
  defrule map_color(%{color: color}=prod,_) do
    {JSONMap,"color"},color_map->
      %{prod| color: color_map[color]}
  end
  defrule map_size(%{size: size}=prod,_) do
    {JSONMap,"size"},size_map->
      %{prod| size: size_map[size]}
  end
  defrule red_is_deleted(%{color: "red"}=prod,_) do
    Dict.put(prod,:deleted,true)
  end
end
result = [
  {:product,%{gender: "male", category: "ipad"}},
  {:product,%{color: "carmine", category: "shirt"}},
  {:product,%{color: "periwinkle", size: "xxl"}}
] |> Adap.Stream.new(ColorPipe) |> Enum.to_list
assert result == [
  {:product,%{gender: "male", category: "ipad"}},
  {:product,%{color: "red", category: "shirt", deleted: true}},
  {:product,%{color: "blue", size: "large"}}
]

Articles
to learn more about the elixir concepts.

Resources
which are currently available to browse on.

mail [email protected] to add your project or resources here 🔥.

FAQ's
to know more about the topic.

mail [email protected] to add your project or resources here 🔥.

Queries
or most google FAQ's about Elixir.

mail [email protected] to add more queries here 🔍.

More Sites
to check out once you're finished browsing here.

0x3d
https://www.0x3d.site/
0x3d is designed for aggregating information.
NodeJS
https://nodejs.0x3d.site/
NodeJS Online Directory
Cross Platform
https://cross-platform.0x3d.site/
Cross Platform Online Directory
Open Source
https://open-source.0x3d.site/
Open Source Online Directory
Analytics
https://analytics.0x3d.site/
Analytics Online Directory
JavaScript
https://javascript.0x3d.site/
JavaScript Online Directory
GoLang
https://golang.0x3d.site/
GoLang Online Directory
Python
https://python.0x3d.site/
Python Online Directory
Swift
https://swift.0x3d.site/
Swift Online Directory
Rust
https://rust.0x3d.site/
Rust Online Directory
Scala
https://scala.0x3d.site/
Scala Online Directory
Ruby
https://ruby.0x3d.site/
Ruby Online Directory
Clojure
https://clojure.0x3d.site/
Clojure Online Directory
Elixir
https://elixir.0x3d.site/
Elixir Online Directory
Elm
https://elm.0x3d.site/
Elm Online Directory
Lua
https://lua.0x3d.site/
Lua Online Directory
C Programming
https://c-programming.0x3d.site/
C Programming Online Directory
C++ Programming
https://cpp-programming.0x3d.site/
C++ Programming Online Directory
R Programming
https://r-programming.0x3d.site/
R Programming Online Directory
Perl
https://perl.0x3d.site/
Perl Online Directory
Java
https://java.0x3d.site/
Java Online Directory
Kotlin
https://kotlin.0x3d.site/
Kotlin Online Directory
PHP
https://php.0x3d.site/
PHP Online Directory
React JS
https://react.0x3d.site/
React JS Online Directory
Angular
https://angular.0x3d.site/
Angular JS Online Directory