Product Features
...
Analytics
Analytics Flows and Processors
Processor Flow Parameters
120min
Refer to the following parameters you will need to configure when you Add Processors and Processor Connections or Create an Analytics Flow.
Generate events by periodically polling specified DB measurement.
- Database: Select database name.
- Filter: SQL filter for the WHERE part of the query.
- Metric Name: Name of the metric for TimeSeriesDB polling.
- Period Secs: Polling interval in seconds.
- Typed: Data typed policy.
Filtering:
- You could add filtering for queries.
- List of fields depends on database and measurement, but for device data, standard filters are: datatype, device_id, is_device_status, register_id, success, tag, topic, value.
Examples:
- Filter by tag: "tag" = 'tag1'
- Filter by tag and success: "tag" = 'tag1' AND "success" = 1
Generate events from specified DataHub device and tag, and attach to output.
- Format: Expected format of message. Possible values: json, fb (flat buffer).
- Topic: Select the device and tag to collect data from.
- Ignore Failed Data (Manufacturing Connect Edge version 3.10.0 and later): Select to ignore data with the output success=false.
- Ignore Null Value (Manufacturing Connect Edge version 3.10.0 and later): Select to ignore data with the output value=null.
Simple generator for testing. Use t as the variable.
- Amplitude Multiplier
- Formula: Supported math functions: sin, cos, tan, power, sqrt, log, exp, abs.
- Generator Type: User defined signal generation or pre-defined generation.
- Periodicity: Periodicity of signal in seconds.
- Strength: Strength of simulated wave approximation - max is 10.
- Timer Interval: Timer in milliseconds. If no event, use this timer to keep displaying the output continually. Enter zero to disable.
Standard Deviation Anomaly Detection (Statistical)
Learn more about Anomaly Detection.
- Topic Reset
- Control Chart Mode?
- Deviations: Number of deviations to keep in range.
- Window Size: Window in which to calculate. Window represents number of seconds.
- Pass Through Value
- Anomaly detection by 3-Sigma Rule is a conventional heuristic used for an approximately normalized data-set.
- A rolling window of values are "observed", and their average and standard deviations are calculated.
- You can define the number of standard deviations that would be considered "normal", hence everything beyond that would be anomalous data.
- In case of an anomaly, the standard deviation window is expanded ever so slightly, so that it can adjust if this anomalous data becomes seasonal.
- This calculation is done because in live data, it is sometimes not preferable for one large anomaly to drastically change the moving average and moving standard deviations.
- With the control chart mode, this calculation can be completely bypassed, if all you need to check is whether the value is within upper/lower limit or not.
- If control chart mode is enabled, there will be no modifications done to the moving window before calculation.
- Currently if you have very small deviations (close to 0), it is difficult to distinguish between anomalous data, so it would be better to have a larger window size for those kinds of data.
- Expected Output Fields: timestamp, current value, moving average, moving standard deviation, upper limit, lower limit, total anomalies, and anomaly field replaces current value, if detected.
- Window Size: Window in which to calculate. Window represents number of seconds.
- Seasonality Size
- Number Of Models: Take the average prediction from how many models.
- Pass Through Value
Learn more about Asset Online Percentage.
- Device Reset
- Start Hour: Start hour when asset online percentage is calculated.
- End Hour: End hour when asset online percentage is calculated.
- Timer Interval: Timer in milliseconds. If no event, use this timer to keep displaying the output continually. Enter zero to disable.
- Pass Through Value
- The input tag connected to this KPI will be online for some time, and offline for other.
- This is indicated by the success field in the incoming message from the tag.
- This KPI is a simple ratio of -> the time the tag has been online vs how long it has been running.
- Inference can be something like the following:
- If the machine is ON for 10 hours, how many hours is it successfully online (running)?
- The timer interval parameter is useful if the connected input tag is offline, but you still want this KPI to "look" for a value every few seconds, defined by the aforementioned timer.
- If you know your tag is going to reliably publish at the expected interval, it is better to disable this timer by entering 0 in the field.
- Expected Output Fields: "asset_online" in percentage, and the current timestamp.
Learn more about Base Conversion.
- Conversion: Which type of conversion?
- Pass Through Value
- With this function, you can convert the base from decimal to binary, decimal to HexaDecimal, decimal to OctaDecimal, and the other way round as well.
- For conversion of various units to decimal, it is expected that the input is a string.
- Your input needs to have a field named value if you are converting from binary/octadecimal/hexadecimal to decimal.
- For example, if 12 is the input in decimal place, the output of decimal -> hexadecimal would be "c".
- For example, if 12 is the input in decimal place, the output of decimal -> binary would be "1100".
Learn more about Change of Value.
- Map Field Name: What field to observe?
- Delta of Tolerance: How much delta of values to ignore? (Helps ignore some error of round off).
- Timer Interval: Timer in milliseconds. If no event, use this timer to keep displaying the output continually. Enter zero to disable.
- Pass Through Value
- Show the entire message payload when the current value is different than the previous value.
- A simple use-case of this KPI is: you want to see a message ONLY if there is a significant change from the previous value, otherwise if things are normal, no output is expected.
- By entering a delta, you can ignore some precision error.
- If you enter delta as 0.0, even the smallest change will be detected by this KPI, or you can enter a delta of 0.1 (for example) if you want to allow ignore a change of +/- 0.1 from your previous value.
- Map Field Name asks you what is the field you want to observe for COV. This can be "value" or "asset_online" etc depending entirely on what is the input message you are sending to this KPI.
- The timer interval parameter is useful if the connected input tag is currently not polling, but you still want this KPI to publish a value every few seconds, defined by the aforementioned timer.
- If you know your input is going to publish at the expected interval, it is better to disable this timer by entering 0 in the field.
- Expected Output: No output if the current value is the same as the previous value (+ or - delta), but outputs the entire input message if there is a change.
Learn more about Combination Processor.
- Combination Type: Combine selected, or combine all fields?
- What Timestamp to Take?: Take timestamp from which input processor?
- Fields to Combine: Enter the fields to combine, with comma as the separator.
- Pass Through Value
- This Processor can be used to combine fields from different input processors, into a single JSON string.
- A definition is needed to be added on the connecting wire after creating this processor.
- The fields to combined, will need to be defined as wire definition_fieldName.
After creating and saving this function processor, you need to add two input processors and connect them to feed the function. See Add a Processor from the Instances Table to learn more. Connection from the first input must be named manufactureEnd and from the second one defectiveUnit. Both connections must be of the Events type. The first one triggers an event at the end of a manufacturing cycle, and the second one triggers an event when a defective unit is discovered.
Learn more about Compliance and Loss.
- Manufacture From: Manufacture End, when value goes from this number.
- Manufacture To: Manufacture End, when value goes to this number.
- Defective From: Defective Unit, when value goes from this number.
- Defective To: Defective Unit, when value goes to this number.
- Timer Interval: Timer in milliseconds. If no event, use this timer to keep displaying the output continually. Enter zero to disable.
- Pass Through Value
Compliance
- If you do not need to keep a track of ALL the failures happening in a factory (like a machine failure/defective units/failing regulation, etc), but if there is a topic(event trigger) which updates whenever there is such an event, you can just monitor how many units are compliant (pass quality assurance/non-defective units).
Production Loss
- If you need to know how many of the manufactured units were unsuccessful, and hence realize the loss incurred on the desired period.
- The timer interval parameter is useful if the connected input tag is offline, but you still want this KPI to "look" for a value every few seconds, defined by the aforementioned timer.
- If you know your tag is going to reliably publish at the expected interval, it is better to disable this timer by entering 0 in the field.
Ready conversions of commonly used Units
Learn more about Conversion.
- Conversion: What type of conversion.
- Pass Through Value
After creating and saving this function processor, you need to add input processor and connect it to feed the function. See Add a Processor from the Instances Table for details. The connection must be named wire definition_fieldName.
Learn more about Expressions.
- Expression: Enter a mathematical expression.
- Output Field Name: Which field name should appear in the output?
- Timer Interval: Timer in milliseconds. If no event, use this timer to keep displaying the output continually. Enter zero to disable.
- Pass Through Value
This function is for creating a mathematical-like expression with your desired topic. For example, you can enter a formula to convert temperature from Celsius to Fahrenheit, or a custom formula to perform on furnace pressure.
- A definition is needed to be added on the connecting wire after creating this KPI. The expression will need to be defined as wire definition_fieldName expression.
- For example, (Celsius_value * 9/5) + 32 will be the expression, and then after creating this expression, you just need to add definition for "Celsius" to the connected input.
- Other examples of how this processor can be used with modifiers such as: + (add), - (subtract), / (divide),* (multiply), ** (raise-to-power), % (modulo division).
- Additionally, comparators such as <** (less than), **>> (greater than), <=** (less than equals), **>= (greater than equals), == (exactly equals) can also be used to obtain a true or false result
- A_value > 10 (implies output will be true if value is greater than 10, otherwise false).
- Regex comparators =~ and !~ are used to compare the left side candidate string to the pattern on the right side.
- Logical operators can be added to the expression as well, with && (logical AND), || (logical OR).
- Ternary operator ? and : are also available for use. For example:
- A_success ? true : false
- In this case, we check if the "left side" of the question mark is true or false, if true, output will be true, otherwise false.
- A_value == 10 ? "equals10" : "notEquals10"
- In this case we are combining expressions and checking if value is equal to 10 and returning output accordingly.
- Finally, there are some useful inbuilt mathematical functions such as:
- sin(), cos(), exp() (exponential), log() (natural logarithm), sqrt() (square root), abs() (absolute value).
- A simple example on how to use them would be: sin(A_value), or sqrt(B_value) + abs(C_value).
- The timer interval parameter is useful if the connected input tag is currently not polling, but you still want this KPI to publish a value every few seconds, defined by the aforementioned timer.
- If you know your input is going to publish at the expected interval, it is better to disable this timer by entering 0 in the field.
- Window Size: Window in which to calculate. Window represents number of seconds.
- Minimum
- Maximum
- Average
- Standard Deviation
- Variance
- Median
- Kurtosis
- Skewness
- Zero Crossing Rate
- Root Mean Square
- Quartiles
- Inter Quartile Range
- Mean Absolute Deviation
- Average Absolute Variation
- Timer Interval: Timer in milliseconds. If no event, use this timer to keep displaying the output continually. Enter zero to disable.
- Pass Through Value
With this function, once you enter the window of values you prefer, you can then select which functions would you like to see in our output.
Let's say you picked a window of N values
- Minimum: Gives you minimum of N values.
- Maximum: Gives you maximum of N values.
- Average: Gives you the average(mean) of N values.
- Standard Deviation: standard deviation = squareRoot( (x - average)^2/(N-1) ).
- Variance: variance = (x - average)^2/(N-1).
- Median: median = N/2 for odd number of window elements, {[N-1] + [N]}/2 for even window elements.
- Kurtosis: kurtosis = [ (N)(N+1) / (N-1)(N-2)(N-3) ] * SIGMA [(x_i - x_avg)/(stdDeviation) ^4].
- Skewness: skewness = [ (N) / (N-1)(N-2)] * SIGMA [(x_i - x_avg)/(stdDeviation) ^3].
- Zero crossing rate: Gives you how many times the signal has crossed zero i.e positive value to negative, OR negative value to positive.
- Root Mean Square: RMS = squareRoot[ SIGMA x^2 / N].
- Quartiles: Divides the signal into 4 equal quartiles, based on medians.
- Inter-quartile Range: Difference between the third quartile and first quartile.
- Mean Absolute Deviation: mean absolute deviation = abs(x - average)/(N).
- Average absolute variation: avg absolute variation = abs(x - average)/(N).
- The timer interval parameter is useful if the connected input tag is currently not polling, but you still want this KPI to publish a value every few seconds, defined by the aforementioned timer.
- If you know your input is going to publish at the expected interval, it is better to disable this timer by entering 0 in the field.
Simple Gaussian Digital Filter
Learn more about Gaussian Filter.
- Window Size: Window in which to calculate. Window represents number of seconds.
- Number of Deviations
- Timer Interval: Timer in milliseconds. If no event, use this timer to keep displaying the output continually. Enter zero to disable.
- Pass Through Value
- First, the mean, standard deviation and variance of a window of values is calculated.
- Coefficient is calculated by the formula 1/ {squareRoot(2 * PI) * deviations}.
- Exponential Part is calculated by the formula exp{ -(1/2) * (current - mean)^2 / (dev)^2 }.
- Multiplying those values gives the Gaussian Function, and then filtered value is obtained.
- The timer interval parameter is useful if the connected input tag is currently not polling, but you still want this KPI to publish a value every few seconds, defined by the aforementioned timer.
- If you know your input is going to publish at the expected interval, it is better to disable this timer by entering 0 in the field.
Calculates the average/mean value from the given multiple inputs.
Learn more about Inputs.
- Processing Type: Data type of the value being processed.
- Field Name: Usually left unchanged. Use it if incoming is message type, and/or non-default "value" field name.
- Pass Through Value
Calculates the maximum value from the given multiple inputs.
Learn more about Inputs.
- Processing Type: Data type of the value being processed.
- Field Name: Usually left unchanged. Use it if incoming is message type, and/or non-default "value" field name.
- Pass Through Value
Calculates the minimum value from the given inputs.
Learn more about Inputs.
- Processing Type: Data type of the value being processed.
- Field Name: Usually left unchanged. Use it if incoming is message type, and/or non-default "value" field name.
- Pass Through Value
Calculates the (cumulative) summation from the given inputs.
Learn more about Inputs.
- Processing Type: Data type of the value being processed.
- Field Name: Usually left unchanged. Use it if incoming is message type, and/or non-default "value" field name.
- Pass Through Value
Learn more about Join Processor.
- Array Sort By
- Array Sort Order: Sort array in ascending order or descending?
- Attach Devices
- Field: Which field to join?
- Input Source: What to show as the input source?
- Map Field Prefix: Any prefix for the map field?
- Output Mode: Output format
- Timestamp Order: Take timestamp in which order?
- Pass Through Value
Output mode
- Two options available here are map and array form.
- Map option is better if standard JSON format is needed (eg. for DataBase output).
- Array option is better if all values are needed in an array format.
Map Field Prefix
- This field can remain empty, but you can add a prefix if you want, in front of the output fields when they are combined.
Array Sort Order
- In array mode, you can chose to arrange field values in either Ascending and Descending order.
Input Source
- What to show as the input source? Device Name, or the connecting wire definition name?
- If Device Name is chosen, it is required to have "deviceName" and "tagName" field in the input.
Linear prediction by ordinary least squares model
- Window Size: How many values to observe before making each prediction?
- Number of Predictions: How many polling intervals in future to predict?
- Polling Interval: What is the polling interval in seconds?
- Timer Interval: Timer in milliseconds. If no event, use this timer to keep displaying the output continually. Enter zero to disable.
- Pass Through Value
- Once window values are full, the mean of values is subtracted from each element and squared (Y_i - Mean_Y)
- Same is done for the time series elements (X_i - Mean_X)
- The sum of squares is sum of (X_i - Mean_X)^2
- The sum of products is sum of (Y_i - Mean_Y) * (X_i - Mean_X)
- The slope is obtained by the formula m = { Sum of products } / { Sum of Squares }
- The intercept is obtained by the formula b = Mean_Y - m * Mean_X
- Finally, prediction is made by extrapolating the number of predictions in the simple formula y = m*X + b
- Residual Error is obtained by modeling for current value, and subtracting the ACTUAL current value
- The timer interval parameter is useful if the connected input tag is currently not polling, but you still want this KPI to publish a value every few seconds, defined by the aforementioned timer.
- If you know your input is going to publish at the expected interval, it is better to disable this timer by entering 0 in the field.
After creating and saving this function processor, you need to add two input processors and connect them to feed the function. See Add a Processor from the Instances Table for details. Connection from the first input must be named maintenance and from the second one failure. Both connections must be of the Events type. The first one triggers an event for maintenance, the second for failure.
Learn more about Maintenance and Failure.
- Maintenance From: Maintenance, when value goes from this number.
- Maintenance To: Maintenance, when value goes to this number.
- Failure From: Failure, when value goes from this number.
- Failure To: Failure, when value goes to this number.
- Unit: Display time since last Maintenance/Failure in Seconds, Minutes, Hours, Days or Weeks.
- Timer Interval: Timer in milliseconds. If no event, use this timer to keep displaying the output continually. Enter zero to disable.
- Pass Through Value
Requirement: A new topic created such that every time there is a maintenance event or a fault/failure, this topic gets updated
- Maintenance events per cycle: You can see the number of maintenance (planned or unplanned) in a production cycle.
- Time since last maintenance: If you know that the machine needs to be checked periodically, you can see time elapsed between 'maintenance' events.
- Timestamp of last maintenance: Keep track of the previous time when maintenance happened.
- Number of faults/failures detected in a period: You can see the number of failures that have happened in a period.
- Time since last failure: If you know that the machine needs to be checked periodically, you can see time elapsed between 'failure' events.
- Timestamp of last failure: Keep track of the previous time when failure happened. The timer interval parameter is useful if the connected input tag is offline, but you still want this KPI to "look" for a value every few seconds, defined by the aforementioned timer.
- If you know your tag is going to reliably publish at the expected interval, it is better to disable this timer by entering 0 in the field.
After creating and saving this function processor, you need to add an input processor and connect it to feed the function. See Add a Processor from the Instances TableAdd a Processor from the Instances Table for details. The connection from the input must be named manufactureEnd. It triggers an event whenever there is end of a manufacture cycle.
- Device Reset
- Unit: What is the unit in which you are entering the ideal Count in? For example, minute, hour, day, week.
- Ideal Count: What is the ideal count of manufacture per hour/day/week?
- From Value: Manufacture End, when value goes from this number.
- To Value: Manufacture End, when value goes to this number.
- Timer Interval: Timer in milliseconds. If no event, use this timer to keep displaying the output continually. Enter zero to disable.
- Pass Through Value
Total Units manufactured KPI
- If you need a simple count of the total number of units manufactured in a period.
Capacity Utilization KPI
You can monitor how much output is being produced, compared to an ideal scenario. Kind of like CPU usage or Memory usage.
- For example, the specifications of a machine says it produces 100 stickers per hour, but currently it is only producing 80 stickers per hour. Hence you can see that the machine is operating only at 80% of its actual capacity.
- Possibly even more useful, if the machine is more than 100% utilization, it can be inferred that the machine is likely to overheat and cause breakdown/failure.
- Note : Capacity Utilization will be updated once manufacture event happens.
- Otherwise, it will be updated when the threshold for the hour/day/week is met (decided through the "unit" parameter).
- The timer interval parameter is useful if the connected input tag is offline, but you still want this KPI to "look" for a value every few seconds, defined by the aforementioned timer.
- If you know your tag is going to reliably publish at the expected interval, it is better to disable this timer by entering 0 in the field.
Simple Moving Average
Learn more about Moving Window.
- Processing Type: Data type of the value being processed.
- Window Size: Size of window in which values are observed.
- Field Name: Usually left unchanged. Use it if incoming is message type, and/or non-default "value" field name.
- Pass Through Value
- Simple moving average, also known as Rolling Mean is just an unweighted average of "n" values, in the desired window.
- The oldest value is then dropped, and a new window with the current value is formed, hence the name "moving average".
Learn more about Moving Window.
- Processing Type: Data type of the value being processed.
- Window Size: Window in which to calculate. Window represents number of seconds.
- Field Name: Usually left unchanged. Use it if incoming is message type, and/or non-default "value" field name.
- Pass Through Value
- Simple moving average, also known as Rolling Mean is just an unweighted average of "n" values, in the desired window.
- The oldest value is then dropped, and a new window with the current value is formed, hence the name "moving average".
Learn more about Moving Window.
- Processing Type: Data type of the value being processed.
- Window Size: Size of window in which values are observed.
- Field Name: Usually left unchanged. Use it if incoming is message type, and/or non-default "value" field name.
- Pass Through Value
- This KPI shows the minimum value from "n" values, in the desired window.
- The oldest value is then dropped, and a new window with the current value is formed, hence the name "moving minimum".
Learn more about Normalization.
- Mode: Which form of Standardization/Normalization.
- Window Size: How many values to observe.
- Pass Through Value
- Initially, a window of value is observed, and the next value is converted between 0 and 1, based on the method of normalization used.
- Afterwards, the values follow moving window algorithm and the next value is again placed between 0 and 1 based on the previous window.
Learn more about Online and Offline Percentage.
- Device Reset
- Reset On Status Change
- Unit: Display Online/Offline in Seconds, Minutes, Hours, Days or Weeks.
- Timer Interval: Timer in milliseconds. If no event, use this timer to keep displaying the output continually. Enter zero to disable.
- Pass Through Value
- If you do not care for how long the sensor has been ON for, but care for how long was it functional, Online/Offline is a raw number to have more detailed knowledge of the time.
- If you enable the "resetOnStatus" parameter, this KPI will only show how long the device has been ON, or how long it has been OFF, and will reset every time the status changes from ON or OFF.
- The timer interval parameter is useful if the connected input tag is offline, but you still want this KPI to "look" for a value every few seconds, defined by the aforementioned timer.
- If you know your tag is going to reliably publish at the expected interval, it is better to disable this timer by entering 0 in the field.
After creating and saving this function processor, you need to add two input processors and connect them to feed the function. See Add a Processor from the Instances TableAdd a Processor from the Instances Table for details. Connection from the first input must be named *manufactureStart *and from the second one - manufactureEnd. Both connections must be of the Values type. The first one triggers an event at the start of a manufacturing cycle, and the second one - at its end.
Learn more about Production Cycle and Cycle Time Radio (CTR).
- Manufacture Start From: Manufacture Start, when value goes from this number.
- Manufacture Start To: Manufacture Start, when value goes to this number.
- Manufacture End From: Manufacture End, when value goes from this number.
- Manufacture End To: Manufacture End, when value goes to this number.
- Ideal Cycle Time: Ideal/Standard production cycle time, in seconds.
- Timer Interval: Timer in milliseconds. If no event, use this timer to keep displaying the output continually. Enter zero to disable.
- Pass Through Value
Actual Production Time KPI
If you want to keep track of how much time it takes to complete production, either by:
- counting the time between when you received an order - to when manufacturing is finished, or
- counting the time between the start of manufacturing - to when inspection ends.
Cycle Time Ratio KPI
If you need to monitor what is the production cycle time compared to an ideal production cycle time
- For example, the current (production) cycle time is 5 times of your ideal cycle time (or) in other words, 'it took you 5 times more time for this production cycle than it actually should/usually does'.
- The timer interval parameter is useful if the connected input tag is offline, but you still want this KPI to "look" for a value every few seconds, defined by the aforementioned timer.
- If you know your tag is going to reliably publish at the expected interval, it is better to disable this timer by entering 0 in the field.
Learn more about Rise and Fall.
- Kpi: Rise or Fall, or both?
- Upper: Upper Threshold
- Lower: Lower Threshold
- Tolerance: Any changes under this tolerance will be ignored.
- Pass Through Value
- Rise: Once you enter a lower threshold and an upper threshold, this function will show you the timestamp at which the current value starts to rise from the lower threshold.
- Fall: Once you enter a lower threshold and an upper threshold, this function will show you the timestamp at which the current value starts to fall from the upper threshold.
- Also provides rise time and fall time in seconds.
Learn more about Rounding.
- Digits: Round to how many decimals (digits)?
- Method: Round to nearest, Round up, Round down?
- Pass Through Value
- Enter the number of digits you would like to round to. For example, entering 2, will give results like 10.22. 10.44 (but not 10.556).
- Entering 0 will ignore all floating point values, and just give the integer value.
- RoundNearest returns the nearest decimal e.g 10.6 gives 11, 10.1 gives 10.
- RoundUp returns the next smallest decimal e.g 10.6 gives 11, 10.1 also gives 11.
- RoundDown returns the previous largest decimal e.g 10.6 gives 10, 10.1 also gives 10.
- RoundEven returns the nearest integer, rounding ties to even (number of digits entered are irrelevant for this case).
Learn more about Signal Decomposition.
- Window Size: How many values in the Periodic interval.
- Model Type: Which type of model.
- Periodicity: The periodic interval.
- Pass Through Value
- Naive version of signal decomposition
- Additive model: Signal = Trend + Seasonality + Residue
- Multiplicative model: Signal = Trend * Seasonality * Residue
- Trend is calculated by linear regression
- Seasonality is calculated by naive differencing
Takes input as a single value, outputs a simple moving window following FIFO algorithm.
- Value Type: What is the type of the incoming value?
- Order: What is the desired order of output window?
- Window Size: Window in which the values will be observed.
- Pass Through Value
- Window Size: Window in which to calculate. Window represents number of seconds.
- Jarque Bera
- Anderson Darling
- Cramer Von Mises
- D Agostino Pearson
- Kolmogorov Smirnov
- Pass Through Value
A few statistical functions and normality tests for a window of values.
Jarque-Bera test:
- This is simple goodness-of-fit test to check if the sample data has similar skewness and kurtosis to a normal distribution.
- The further the JB tet value is from zero, the stronger the fact that the data does not belong to the normal distribution.
- Skewness and kurtosis are calculated from the window of data and then JB value is calculated.
- JB = (n/6) (skewness^2 + (1/4)(kurtosis-3)^2)
- P-value is calculated by subtracting cumulative distribution function at the JB value for the ChiSquared distribution with 1.
Cramer-Von-Mises test:
- This is goodness-of-fit test which uses the summed squared differences between the sample of data, and the expected cumulative distribution function.
- First, a window of data is collected, then mean and standard deviation is calculated.
- Data is then sorted, and Z-scores are calculated by standardizing the data.
- cramerVonMises = (1/12n) + SIGMA { ((2i-1)/(2n) - phiZ)^2 }
- P-value is calculated, based on the value of cramerVonMises.
Anderson-Darling test:
- This is a statistical test to check whether a given sample of data is from an empirical distribution function or not.
- In this case, we are using a normal distribution - meaning, we use this test to see how far the data departs from an ideal normal distribution.
- Compared to other tests, Anderson-Darling test gives more weight to the tails of the distribution
- First, a window of data is collected, then mean and standard deviation is calculated.
- Data is then sorted, and Z-scores are calculated by standardizing the data.
- A^2 = -n - (1/n)* SIGMA{ (2i-1)ln(phiZ[i]) + (2(n-i)+1)ln(phZ[i]) } formula is applied, where phiZ is the cumulative distribution function for the Z-scores.
- P-value is calculated, based on the value of A^2.
D'Agostino Pearson test:
- With a combination of skewness and kurtosis test, D'Agostino Pearson test checks whether the shape of the window of values matches a normal distributions.
Kolmogorov-Smirnov test:
- Lilliefors test is use to check the hypothesis of normality for the Kolmogorov-Smirnov test.
- First, a window of data is collected, then mean and standard deviation is calculated.
- D+ and D- are calculated by taking maximum discrepancy between the empirical distribution function and the cumulative distribution function.
- K statistic is just max(D+,D-)*Sqrt(n).
Statistical Prediction by Fourier extrapolation
Learn more about Statistical Prediction.
- Display Prediction On Current Timestamp
- Window Size: How many values to observe before making each prediction? (assumed with 1 second as poling interval).
- Number of Predictions: How many seconds in future to predict? (assumed with 1 second as poling interval).
- Frequency Difference: Frequency sampling distance in seconds.
- Number of Harmonics
- Pass Through Value
- To achieve a prediction of value in the future, Fast fourier transform (FFT) is done on the time series signal.
- This FFT is then used to de-trend the data, generate sine-cosine waveform out of the signal, and then the signal is restored to its best capacity.
- The frequency difference represents the sampling rate of discretization.
- For example, if the window has 10 elements, 1 second frequency difference will generate 10 frequency samples (maximum). 2 seconds will generate 5 samples, and so on.
- Usually 1 is good, because it is a time series signal, so you're sampling at every second.
- Harmonics represent the number of waves to "add" to the fundamental wave, to reach the desired signal.
- Some signals have a huge number of harmonics, for example square waves have theoretically infinite harmonics.
- For the number of harmonics, it is better to start with a lower number and play around with it to see what best fits your needs, rather than just entering a high number.
- Due to limitations of working with live data, it is preferable to have a window of data to "observe" and then give a "prediction" for a future time.
- For now, the poling interval of the subscription tag is assumed to be 1 second.
- Expected Output: Until the window is filled up, no output is expected. Afterwards, the prediction, with the timestamp of prediction will be the output.
After creating and saving this function processor, you need to the number of input processors defined by the Number of Inputs value and connect them to feed the function. See Add a Processor from the Instances TableAdd a Processor from the Instances Table for details. The connections must be of the Values type, named numerically starting from 0 (0, 1, 2, etc.).
Since TensorFlow 1.6, this processor option may not be available when running on an older CPU.
Scripting for data manipulations
Learn more about Tengo Script.
- Timeout: timeout in ms
- Pass Through Value
- This processor uses Tengo scripting, with its rules similar to GoLang.
- Simple Use case: The output you receive from DeviceHub message or from Analytics, might not be helpful, and you want to change the name of a field within that message.
- A few examples:
- Instead of "value" you want your data to show "furnace pressure".
- Instead of having all fields of the message, you only want to have "success", "deviceId" and "value" and skip everything else.
- You want to add a custom field to the message, based on the "value" field, such as: if value = 2, "error": "overheating".
- You only want to change the timestamp to human readable format instead of Unix timestamp, but keep otherwise keep the message as-it-is.
- In general, if you want to add, remove or modify ANY field in the input, this scripting processor is very useful.
- Single Input: Every message from the input, is stored as an array called "values".
- Using "for i, message in values{}" statement, allows to access this input and manipulate it.
- Here "i" represents what is the definition defined in the connection wire, more useful for multi-inputs, to identify and distinguish between each inputs, so for single input, "i" can be replaced with "_".
- "message" contains the entire message in the input, and you can perform your manipulations to this "message" parameter.
- Note: "i" and "message" can be any custom variable name.
- Multiple Input: Suppose you have 3 different inputs attached to this scripting processor, with the connecting wires named A, B and C.
- "for i, message in values{}" will now have 3 inputs.
- "i" represents from which connection wire is the input coming from.
- with if-else statements, every input can be manipulated in isolation, or a common manipulation can be applied to all inputs.
- You can choose to output "result" only for one input, or all of the inputs.
- Miscellaneous information:
After creating and saving this function processor, you need to the number of input processors defined by the Number of Inputs value and connect them to feed the function. See Add a Processor from the Instances TableAdd a Processor from the Instances Table for details. The connections must be of the Values type, named numerically starting from 0 (0, 1, 2, and so on).
Since TensorFlow 1.6, this processor option may not be available when running on an older CPU.
Processor for feeding images to an already created TensorFlow model.
- Model Path
- Tags: Tags for loading model. If multiple tags, use comma separator.
- Input Operation: Input tensor used while creating the model. Usually the dictionary of values you feed to the model.
- Output Operation: Output tensor used while creating the model. Usually the output of the model
- Gray
- Image Format
- Time Shift Ms: By how many milliseconds to shift the current timestamp.
- Pass Through Value
Processor for feeding time-series data to an already created TensorFlow model.
- Number of Inputs
- Tags: Tags for loading model. If there are multiple tags, use comma separator.
- Input Operation: Input tensor used while creating the model. Usually the dictionary of values you feed to the model.
- Output Operation: Output tensor used while creating the model. Usually the output of the model.
- Window Size: SIze of values to process. This is an array of values that is modified for every new value that arrives.
- Time Shift Ms: By how many milliseconds to shift the current timestamp.
If you would like to compare your current value with a desired value, to count as UpTime or DownTime.
Learn more about Uptime and Downtime by Value.
- Device Reset
- Unit: Display UpTime/DownTime in Seconds, Minutes, Hours, Days or Weeks.
- Condition: What sort of comparison to make; Downtime if current value is greater(>), less(<), equal(==) or notEqual(!=) than desired value.
- Desired Value: Desired value to check current values with.
- Delta of Tolerance: Allow freedom of desiredValue + or - this delta.
- Timer Interval: Timer in milliseconds in which to push values. Zero disables this timer.
- Pass Through Value
- By setting the condition, you can define: Downtime if current value is greater(>), less(<), equal(==) or notEqual(!=) than desired value.
- You can enter a delta too, if you want to forgive some sort of round off error.
- The timer interval parameter is useful if the connected input tag is offline, but you still want this KPI to "look" for a value every few seconds, defined by the aforementioned timer.
- If you know your tag is going to reliably publish at the expected interval, it is better to disable this timer by entering 0 in the field.
Learn more about XMR Chart.
- Window Size: Window in which to calculate. Window represents number of seconds.
- Process Limit Multiplier
- Timer Interval: Timer in milliseconds in which to push values. Zero disables this timer.
- Pass Through Value
- Once the window is filled with values, a central line is calculated which remains constant moving forward.
- Other fields that are calculated once window is filled are:
- Moving ranges : Difference of successive values.
- Average moving range : Average of the moving ranges.
- Process Limit Multiplier is used to calculate the upper and lower process limits.
- Upper and Lower Natural Process limit : Calculated by taking the product of process limit multiplier and average moving range, and adding/subtracting it to the central line.
- In some cases, the central line is adjusted. Here are the cases:
- If current value is outside the process limits, multiple times in a row.
- If the multiple values in a row are closer to either limits, than they are to the central line.
- The timer interval parameter is useful if the connected input tag is currently not polling, but you still want this KPI to publish a value every few seconds, defined by the aforementioned timer.
- If you know your input is going to publish at the expected interval, it is better to disable this timer by entering 0 in the field.
Publish all inputs to a specified DB metric.
- Database: If empty - will use default database.
- Measurement: Name of the measurement to publish.
Note: The Measurement parameter is a required field.
- Typed: Publish typed name.
Publish events from input to a specified DataHub topic.
- Single Topic
- Topic: Topic name for DataHub publish.
Standard output of data into a console.