When developing consumer applications that make use of Precision Time Protocol (PTP) events on a bare-metal cluster node, you deploy your consumer application in a separate application pod. The consumer application subscribes to PTP events by using the PTP events REST API v2.
The following information provides general guidance for developing consumer applications that use PTP events. A complete events consumer application example is outside the scope of this information. |
Use the Precision Time Protocol (PTP) fast event REST API v2 to subscribe cluster applications to PTP events that the bare-metal cluster node generates.
The fast events notifications framework uses a REST API for communication. The PTP events REST API v1 and v2 are based on the O-RAN O-Cloud Notification API Specification for Event Consumers 3.0 that is available from O-RAN ALLIANCE Specifications. Only the PTP events REST API v2 is O-RAN v3 compliant. |
Applications subscribe to PTP events by using an O-RAN v3 compatible REST API in the producer-side cloud event proxy sidecar.
The cloud-event-proxy
sidecar container can access the same resources as the primary application container without using any of the resources of the primary application and with no significant latency.
The linuxptp-daemon
process in the PTP Operator-managed pod runs as a Kubernetes DaemonSet
and manages the various linuxptp
processes (ptp4l
, phc2sys
, and optionally for grandmaster clocks, ts2phc
).
The linuxptp-daemon
passes the event to the UNIX domain socket.
The PTP plugin reads the event from the UNIX domain socket and passes it to the cloud-event-proxy
sidecar in the PTP Operator-managed pod.
cloud-event-proxy
delivers the event from the Kubernetes infrastructure to Cloud-Native Network Functions (CNFs) with low latency.
The cloud-event-proxy
sidecar in the PTP Operator-managed pod processes the event and publishes the event by using the PTP events REST API v2.
The consumer application sends an API request to the producer cloud-event-proxy
sidecar to create a PTP events subscription.
Once subscribed, the consumer application listens to the address specified in the resource qualifier and receives and processes the PTP events.
To start using PTP fast event notifications for a network interface in your cluster, you must enable the fast event publisher in the PTP Operator PtpOperatorConfig
custom resource (CR) and configure ptpClockThreshold
values in a PtpConfig
CR that you create.
You have installed the OKD CLI (oc
).
You have logged in as a user with cluster-admin
privileges.
You have installed the PTP Operator.
Modify the default PTP Operator config to enable PTP fast events.
Save the following YAML in the ptp-operatorconfig.yaml
file:
apiVersion: ptp.openshift.io/v1
kind: PtpOperatorConfig
metadata:
name: default
namespace: openshift-ptp
spec:
daemonNodeSelector:
node-role.kubernetes.io/worker: ""
ptpEventConfig:
apiVersion: 2.0 (1)
enableEventPublisher: true (2)
1 | Enable the PTP events REST API v2 for the PTP event producer by setting the ptpEventConfig.apiVersion to 2.0.
The default value is 1.0. |
2 | Enable PTP fast event notifications by setting enableEventPublisher to true . |
In OKD 4.13 or later, you do not need to set the |
Update the PtpOperatorConfig
CR:
$ oc apply -f ptp-operatorconfig.yaml
Create a PtpConfig
custom resource (CR) for the PTP enabled interface, and set the required values for ptpClockThreshold
and ptp4lOpts
.
The following YAML illustrates the required values that you must set in the PtpConfig
CR:
spec:
profile:
- name: "profile1"
interface: "enp5s0f0"
ptp4lOpts: "-2 -s --summary_interval -4" (1)
phc2sysOpts: "-a -r -m -n 24 -N 8 -R 16" (2)
ptp4lConf: "" (3)
ptpClockThreshold: (4)
holdOverTimeout: 5
maxOffsetThreshold: 100
minOffsetThreshold: -100
1 | Append --summary_interval -4 to use PTP fast events. |
2 | Required phc2sysOpts values. -m prints messages to stdout . The linuxptp-daemon DaemonSet parses the logs and generates Prometheus metrics. |
3 | Specify a string that contains the configuration to replace the default /etc/ptp4l.conf file. To use the default configuration, leave the field empty. |
4 | Optional. If the ptpClockThreshold stanza is not present, default values are used for the ptpClockThreshold fields. The stanza shows default ptpClockThreshold values. The ptpClockThreshold values configure how long after the PTP master clock is disconnected before PTP events are triggered. holdOverTimeout is the time value in seconds before the PTP clock event state changes to FREERUN when the PTP master clock is disconnected. The maxOffsetThreshold and minOffsetThreshold settings configure offset values in nanoseconds that compare against the values for CLOCK_REALTIME (phc2sys ) or master offset (ptp4l ). When the ptp4l or phc2sys offset value is outside this range, the PTP clock state is set to FREERUN . When the offset value is within this range, the PTP clock state is set to LOCKED . |
For a complete example CR that configures linuxptp
services as an ordinary clock with PTP fast events, see Configuring linuxptp services as ordinary clock.
PTP event consumer applications require the following features:
A web service running with a POST
handler to receive the cloud native PTP events JSON payload
A createSubscription
function to subscribe to the PTP events producer
A getCurrentState
function to poll the current state of the PTP events producer
The following example Go snippets illustrate these requirements:
func server() {
http.HandleFunc("/event", getEvent)
http.ListenAndServe(":9043", nil)
}
func getEvent(w http.ResponseWriter, req *http.Request) {
defer req.Body.Close()
bodyBytes, err := io.ReadAll(req.Body)
if err != nil {
log.Errorf("error reading event %v", err)
}
e := string(bodyBytes)
if e != "" {
processEvent(bodyBytes)
log.Infof("received event %s", string(bodyBytes))
} else {
w.WriteHeader(http.StatusNoContent)
}
}
import (
"github.com/redhat-cne/sdk-go/pkg/pubsub"
"github.com/redhat-cne/sdk-go/pkg/types"
v1pubsub "github.com/redhat-cne/sdk-go/v1/pubsub"
)
// Subscribe to PTP events using v2 REST API
s1,_:=createsubscription("/cluster/node/<node_name>/sync/sync-status/sync-state")
s2,_:=createsubscription("/cluster/node/<node_name>/sync/ptp-status/lock-state")
s3,_:=createsubscription("/cluster/node/<node_name>/sync/gnss-status/gnss-sync-status")
s4,_:=createsubscription("/cluster/node/<node_name>/sync/sync-status/os-clock-sync-state")
s5,_:=createsubscription("/cluster/node/<node_name>/sync/ptp-status/clock-class")
// Create PTP event subscriptions POST
func createSubscription(resourceAddress string) (sub pubsub.PubSub, err error) {
var status int
apiPath := "/api/ocloudNotifications/v2/"
localAPIAddr := "localhost:8989" // vDU service API address
apiAddr := "ptp-event-publisher-service-<node_name>.openshift-ptp.svc.cluster.local:9043" (1)
apiVersion := "2.0"
subURL := &types.URI{URL: url.URL{Scheme: "http",
Host: apiAddr
Path: fmt.Sprintf("%s%s", apiPath, "subscriptions")}}
endpointURL := &types.URI{URL: url.URL{Scheme: "http",
Host: localAPIAddr,
Path: "event"}}
sub = v1pubsub.NewPubSub(endpointURL, resourceAddress, apiVersion)
var subB []byte
if subB, err = json.Marshal(&sub); err == nil {
rc := restclient.New()
if status, subB = rc.PostWithReturn(subURL, subB); status != http.StatusCreated {
err = fmt.Errorf("error in subscription creation api at %s, returned status %d", subURL, status)
} else {
err = json.Unmarshal(subB, &sub)
}
} else {
err = fmt.Errorf("failed to marshal subscription for %s", resourceAddress)
}
return
}
1 | Replace <node_name> with the FQDN of the node that is generating the PTP events. For example, compute-1.example.com . |
//Get PTP event state for the resource
func getCurrentState(resource string) {
//Create publisher
url := &types.URI{URL: url.URL{Scheme: "http",
Host: "ptp-event-publisher-service-<node_name>.openshift-ptp.svc.cluster.local:9043", (1)
Path: fmt.SPrintf("/api/ocloudNotifications/v2/%s/CurrentState",resource}}
rc := restclient.New()
status, event := rc.Get(url)
if status != http.StatusOK {
log.Errorf("CurrentState:error %d from url %s, %s", status, url.String(), event)
} else {
log.Debugf("Got CurrentState: %s ", event)
}
}
1 | Replace <node_name> with the FQDN of the node that is generating the PTP events. For example, compute-1.example.com . |
Use the following example PTP event consumer custom resources (CRs) as a reference when deploying your PTP events consumer application for use with the PTP events REST API v2.
apiVersion: v1
kind: Namespace
metadata:
name: cloud-events
labels:
security.openshift.io/scc.podSecurityLabelSync: "false"
pod-security.kubernetes.io/audit: "privileged"
pod-security.kubernetes.io/enforce: "privileged"
pod-security.kubernetes.io/warn: "privileged"
name: cloud-events
openshift.io/cluster-monitoring: "true"
annotations:
workload.openshift.io/allowed: management
apiVersion: apps/v1
kind: Deployment
metadata:
name: cloud-consumer-deployment
namespace: cloud-events
labels:
app: consumer
spec:
replicas: 1
selector:
matchLabels:
app: consumer
template:
metadata:
annotations:
target.workload.openshift.io/management: '{"effect": "PreferredDuringScheduling"}'
labels:
app: consumer
spec:
nodeSelector:
node-role.kubernetes.io/worker: ""
serviceAccountName: consumer-sa
containers:
- name: cloud-event-consumer
image: cloud-event-consumer
imagePullPolicy: Always
args:
- "--local-api-addr=consumer-events-subscription-service.cloud-events.svc.cluster.local:9043"
- "--api-path=/api/ocloudNotifications/v2/"
- "--api-addr=127.0.0.1:8089"
- "--api-version=2.0"
- "--http-event-publishers=ptp-event-publisher-service-NODE_NAME.openshift-ptp.svc.cluster.local:9043"
env:
- name: NODE_NAME
valueFrom:
fieldRef:
fieldPath: spec.nodeName
- name: CONSUMER_TYPE
value: "PTP"
- name: ENABLE_STATUS_CHECK
value: "true"
volumes:
- name: pubsubstore
emptyDir: {}
apiVersion: v1
kind: ServiceAccount
metadata:
name: consumer-sa
namespace: cloud-events
apiVersion: v1
kind: Service
metadata:
annotations:
prometheus.io/scrape: "true"
name: consumer-events-subscription-service
namespace: cloud-events
labels:
app: consumer-service
spec:
ports:
- name: sub-port
port: 9043
selector:
app: consumer
sessionAffinity: None
type: ClusterIP
Deploy your cloud-event-consumer
application container and subscribe the cloud-event-consumer
application to PTP events posted by the cloud-event-proxy
container in the pod managed by the PTP Operator.
Subscribe consumer applications to PTP events by sending a POST
request to http://ptp-event-publisher-service-NODE_NAME.openshift-ptp.svc.cluster.local:9043/api/ocloudNotifications/v2/subscriptions
passing the appropriate subscription request payload.
|
Verify that the cloud-event-consumer
container in the application pod is receiving Precision Time Protocol (PTP) events.
You have installed the OpenShift CLI (oc
).
You have logged in as a user with cluster-admin
privileges.
You have installed and configured the PTP Operator.
You have deployed a cloud events application pod and PTP events consumer application.
Check the logs for the deployed events consumer application. For example, run the following command:
$ oc -n cloud-events logs -f deployment/cloud-consumer-deployment
time = "2024-09-02T13:49:01Z"
level = info msg = "transport host path is set to ptp-event-publisher-service-compute-1.openshift-ptp.svc.cluster.local:9043"
time = "2024-09-02T13:49:01Z"
level = info msg = "apiVersion=2.0, updated apiAddr=ptp-event-publisher-service-compute-1.openshift-ptp.svc.cluster.local:9043, apiPath=/api/ocloudNotifications/v2/"
time = "2024-09-02T13:49:01Z"
level = info msg = "Starting local API listening to :9043"
time = "2024-09-02T13:49:06Z"
level = info msg = "transport host path is set to ptp-event-publisher-service-compute-1.openshift-ptp.svc.cluster.local:9043"
time = "2024-09-02T13:49:06Z"
level = info msg = "checking for rest service health"
time = "2024-09-02T13:49:06Z"
level = info msg = "health check http://ptp-event-publisher-service-compute-1.openshift-ptp.svc.cluster.local:9043/api/ocloudNotifications/v2/health"
time = "2024-09-02T13:49:07Z"
level = info msg = "rest service returned healthy status"
time = "2024-09-02T13:49:07Z"
level = info msg = "healthy publisher; subscribing to events"
time = "2024-09-02T13:49:07Z"
level = info msg = "received event {\"specversion\":\"1.0\",\"id\":\"ab423275-f65d-4760-97af-5b0b846605e4\",\"source\":\"/sync/ptp-status/clock-class\",\"type\":\"event.sync.ptp-status.ptp-clock-class-change\",\"time\":\"2024-09-02T13:49:07.226494483Z\",\"data\":{\"version\":\"1.0\",\"values\":[{\"ResourceAddress\":\"/cluster/node/compute-1.example.com/ptp-not-set\",\"data_type\":\"metric\",\"value_type\":\"decimal64.3\",\"value\":\"0\"}]}}"
Optional. Test the REST API by using oc
and port-forwarding port 9043
from the linuxptp-daemon
deployment.
For example, run the following command:
$ oc port-forward -n openshift-ptp ds/linuxptp-daemon 9043:9043
Forwarding from 127.0.0.1:9043 -> 9043
Forwarding from [::1]:9043 -> 9043
Handling connection for 9043
Open a new shell prompt and test the REST API v2 endpoints:
$ curl -X GET http://localhost:9043/api/ocloudNotifications/v2/health
OK
You can monitor PTP fast events metrics from cluster nodes where the linuxptp-daemon
is running.
You can also monitor PTP fast event metrics in the OKD web console by using the preconfigured and self-updating Prometheus monitoring stack.
Install the OKD CLI oc
.
Log in as a user with cluster-admin
privileges.
Install and configure the PTP Operator on a node with PTP-capable hardware.
Start a debug pod for the node by running the following command:
$ oc debug node/<node_name>
Check for PTP metrics exposed by the linuxptp-daemon
container. For example, run the following command:
sh-4.4# curl http://localhost:9091/metrics
# HELP cne_api_events_published Metric to get number of events published by the rest api
# TYPE cne_api_events_published gauge
cne_api_events_published{address="/cluster/node/compute-1.example.com/sync/gnss-status/gnss-sync-status",status="success"} 1
cne_api_events_published{address="/cluster/node/compute-1.example.com/sync/ptp-status/lock-state",status="success"} 94
cne_api_events_published{address="/cluster/node/compute-1.example.com/sync/ptp-status/class-change",status="success"} 18
cne_api_events_published{address="/cluster/node/compute-1.example.com/sync/sync-status/os-clock-sync-state",status="success"} 27
Optional.
You can also find PTP events in the logs for the cloud-event-proxy
container.
For example, run the following command:
$ oc logs -f linuxptp-daemon-cvgr6 -n openshift-ptp -c cloud-event-proxy
To view the PTP event in the OKD web console, copy the name of the PTP metric you want to query, for example, openshift_ptp_offset_ns
.
In the OKD web console, click Observe → Metrics.
Paste the PTP metric name into the Expression field, and click Run queries.
The following table describes the PTP fast events metrics that are available from cluster nodes where the linuxptp-daemon
service is running.
Metric | Description | Example |
---|---|---|
|
Returns the PTP clock class for the interface.
Possible values for PTP clock class are 6 ( |
|
|
Returns the current PTP clock state for the interface.
Possible values for PTP clock state are |
|
|
Returns the delay in nanoseconds between the primary clock sending the timing packet and the secondary clock receiving the timing packet. |
|
|
Returns the current status of the highly available system clock when there are multiple time sources on different NICs.
Possible values are 0 ( |
|
|
Returns the frequency adjustment in nanoseconds between 2 PTP clocks.
For example, between the upstream clock and the NIC, between the system clock and the NIC, or between the PTP hardware clock ( |
|
|
Returns the configured PTP clock role for the interface.
Possible values are 0 ( |
|
|
Returns the maximum offset in nanoseconds between 2 clocks or interfaces.
For example, between the upstream GNSS clock and the NIC ( |
|
|
Returns the offset in nanoseconds between the DPLL clock or the GNSS clock source and the NIC hardware clock. |
|
|
Returns a count of the number of times the |
|
|
Returns a status code that shows whether the PTP processes are running or not. |
|
|
Returns values for
|
|
The following table describes the PTP fast event metrics that are available only when PTP grandmaster clock (T-GM) is enabled.
Metric | Description | Example |
---|---|---|
|
Returns the current status of the digital phase-locked loop (DPLL) frequency for the NIC.
Possible values are -1 ( |
|
|
Returns the current status of the NMEA connection.
NMEA is the protocol that is used for 1PPS NIC connections.
Possible values are 0 ( |
|
|
Returns the status of the DPLL phase for the NIC.
Possible values are -1 ( |
|
|
Returns the current status of the NIC 1PPS connection.
You use the 1PPS connection to synchronize timing between connected NICs.
Possible values are 0 ( |
|
|
Returns the current status of the global navigation satellite system (GNSS) connection.
GNSS provides satellite-based positioning, navigation, and timing services globally.
Possible values are 0 ( |
|