The authors consider the problem of compressive sensed video recovery via iterative thresholding algorithm. Traditionally, it is assumed that some fixed sparsifying transform is applied at each iteration of the algorithm. In order to improve the recovery performance, at each iteration the thresholding could be applied for different transforms in order to obtain several estimates for each pixel. Then the resulting pixel value is computed based on obtained estimates using simple averaging. However, calculation of the estimates leads to significant increase in reconstruction complexity. Therefore, the authors propose a heuristic approach, where at each iteration only one transform is randomly selected from some set of transforms. First, they present simple examples, when block-based 2D discrete cosine transform is used as the sparsifying transform, and show that the random selection of the block size at each iteration significantly outperforms the case when fixed block size is used. Second, building on these simple examples, they apply the proposed approach when video block-matching and 3D filtering (VBM3D) is used for the thresholding and show that the random transform selection within VBM3D allows to improve the recovery performance as compared with the recovery based on VBM3D with fixed transform.

In this work, we consider the problem of jointly minimizing the average cost of sampling and transmitting status updates by users over a wireless channel subject to average Age of Information (AoI) constraints. Errors in the transmission may occur and a policy has to decide if the users sample a new packet or attempt to retransmission the packet sampled previously. The cost consists of both sampling and transmission costs. The sampling of a new packet after a failure imposes an additional cost on the system. We formulate a stochastic optimization problem with the average cost in the objective under average AoI constraints. To solve this problem, we propose three scheduling policies: (a) a dynamic policy, which is centralized and requires full knowledge of the state of the system and (b) two stationary randomized policies that require no knowledge of the state of the system. We utilize tools from Lyapunov optimization theory and Discrete-Time Markov Chain (DTMC) to provide the dynamic policy and the randomized ones, respectively. Simulation results show the importance of providing the option to transmit an old packet in order to minimize the total average cost.

We consider the problem of minimizing the time average cost of sampling and transmitting status updates by users over a wireless channel subject to average Age of Information constraints (AoI). Errors in the transmission may occur and the scheduling algorithm has to decide if the users sample a new packet or attempt for retransmission of the packet sampled previously. The cost consists of both sampling and transmission costs. The sampling of a new packet after a failure imposes an additional cost in the system. We formulate a stochastic optimization problem with time average cost in the objective under time average AoI constraints. To solve this problem, we apply tools from Lyapunov optimization theory and develop a dynamic algorithm that takes decisions in a slot-by-slot basis. The algorithm decides if a user: a) samples a new packet, b) transmits the old one, c) remains silent. We provide optimality guarantees of the algorithm and study its performance in terms of time average cost and AoI through simulation results.

We consider an IoT sensing network with multiple users, multiple energy harvesting sensors, and a wireless edge node acting as a gateway between the users and sensors. The users request for updates about the value of physical processes, each of which is measured by one sensor. The edge node has a cache storage that stores the most recently received measurements from each sensor. Upon receiving a request, the edge node can either command the corresponding sensor to send a status update, or use the data in the cache. We aim to find the best action of the edge node to minimize the average long-term cost which trade-offs between the age of information and energy consumption. We propose a practical reinforcement learning approach that finds an optimal policy without knowing the exact battery levels of the sensors. Simulation results show that the proposed method significantly reduces the average cost compared to several baseline methods.

We consider a resource-constrained IoT network, where multiple users make on-demand requests to a cache-enabled edge node to send status updates about various random processes, each monitored by an energy harvesting sensor. The edge node serves users requests by deciding whether to command the corresponding sensor to send a fresh status update or retrieve the most recently received measurement from the cache. Our objective is to find the best actions of the edge node to minimize the average age of information (AoI) of the received measurements upon request, i.e., average on-demand AoI, subject to per-slot transmission and energy constraints. First, we derive a Markov decision process model and propose an iterative algorithm that obtains an optimal policy. Then, we develop an asymptotically optimal low-complexity algorithm - termed relax-then-truncate - and prove that it is optimal as the number of sensors goes to infinity. Simulation results illustrate that the proposed relax-then-truncate approach significantly reduces the average on-demand AoI compared to a request-aware greedy policy and a weighted AoI policy, and also depict that it performs close to the optimal solution even for moderate numbers of sensors.

Information freshness is crucial for time-critical IoT applications, e.g., monitoring and control. We consider an IoT status update system with users, energy harvesting sensors, and a cache-enabled edge node. The users receive time-sensitive information about physical quantities, each measured by a sensor. Users demand for the information from the edge node whose cache stores the most recently received measurements from each sensor. To serve a request, the edge node either commands the sensor to send an update or retrieves the aged measurement from the cache. We aim at finding the best actions of the edge node to minimize the average AoI of the served measurements at the users, termed on-demand AoI. We model this problem as a Markov decision process and develop reinforcement learning (RL) algorithms: model-based value iteration and model-free Q-learning. We also propose a Q-learning method for the realistic case where the edge node is informed about the sensors battery levels only via the status updates. The case under transmission limitations is also addressed. Furthermore, properties of an optimal policy are characterized. Simulation results show that an optimal policy is a threshold-based policy and that the proposed RL methods significantly reduce the average cost compared to several baselines.

Delivering timely status information of a random process has become increasingly important for time-sensitive applications, e.g., vehicle tracking and environment monitoring. We consider an IoT sensing network, where a cache-enabled wireless edge node receives on-demand requests from multiple users to send status updates on physical quantities, each measured by an energy harvesting sensor. To serve users' requests, the edge node uses the current information state (i.e., the number of requests, battery level, and AoI for each sensor) to decide whether to command a sensor to send a status update or to retrieve the most recently received sensor's measurements from the cache. We aim at finding the best actions of the edge node to minimize the average AoI of the served measurements at the users, i.e., average on-demand AoI. We model this as a Markov decision process problem and derive a relative value iteration algorithm to find an optimal policy. Simulation results illustrate the threshold-based structure of an optimal policy and show that the proposed on-demand updating policy outperforms the greedy (myopic) policy and also, by accounting for the per-sensor request frequencies and intensities, the pure average AoI minimization policy that keeps the edge node updated regardless of requests.

Content caching is a promising approach to reduce data traffic in the back-haul links. We consider a system where multiple users request items from a cache-enabled base station that is connected to a cloud. The users request items according to the user preferences in a time-dependent fashion, i.e., a user is likely to request the next chunk (item) of the file requested at a previous time slot. Whenever the requested item is not in the cache, the base station downloads it from the cloud and forwards it to the user. In the meanwhile, the base station decides whether to replace one item in the cache by the fetched item, or to discard it. We model the problem as a Markov decision process (MDP) and propose a novel state space that takes advantage of the dynamics of the users requests. We use reinforcement learning and propose a Q-learning algorithm to find an optimal cache replacement policy that maximizes the cache hit ratio without knowing the popularity profile distribution, probability distribution of items, and user preference model. Simulation results show that the proposed algorithm improves the cache hit ratio compared to other baseline policies.

We consider status updating under inexact knowledge of the battery level of an energy harvesting (EH) sensor that sends status updates about a random process to users via a cache-enabled edge node. More precisely, the control decisions are performed by relying only on the battery level knowledge captured from the last received status update packet. Upon receiving on demand requests for fresh information from the users, the edge node uses the available information to decide whether to command the sensor to send a status update or to retrieve the most recently received measurement from the cache. We seek for the best actions of the edge node to minimize the average AoI of the served measurements, i.e., average on-demand AoI. Accounting for the partial battery knowledge, we model the problem as a partially observable Markov decision process (POMDP), and, through characterizing its key structures, develop a dynamic programming algorithm to obtain an optimal policy. Simulation results illustrate the threshold-based structure of an optimal policy and show the gains obtained by the proposed optimal POMDP-based policy compared to a request-aware greedy (myopic) policy.

We consider the problem of channel estimation in hybrid transceiver architectures operating in millimeter wave (mmWave) band. Due to the dynamic features of the environment and the sensitivity of mmWave bands to blockage and deafness, it is important to estimate mmWave channels with a low complexity and high performance algorithm. In this regard, we exploit the sparse structure of the frequency-selective mmWave channels and formulate the channel estimation problem as a sparse signal reconstruction in frequency domain. In order to solve the estimation problem, we propose a multi-stage based low complexity algorithm. Simulation results show that the proposed algorithm significantly reduces the computational complexity while preserving the quality of the estimation.

Sparse signals, encountered in many wireless and signal acquisition applications, can be acquired via compressed sensing (CS) to reduce computations and transmissions, crucial for resource-limited devices, e.g., wireless sensors. Since the information signals are often continuous-valued, digital communication of compressive measurements requires quantization. In such a quantized compressed sensing (QCS) context, we address remote acquisition of a sparse source through vector quantized noisy compressive measurements. We propose a deep encoder-decoder architecture, consisting of an encoder deep neural network (DNN), a quantizer, and a decoder DNN, that realizes low-complexity vector quantization aiming at minimizing the mean-square error of the signal reconstruction for a given quantization rate. We devise a supervised learning method using stochastic gradient descent and backpropagation to train the system blocks. Strategies to overcome the vanishing gradient problem are proposed. Simulation results show that the proposed non-iterative DNN-based QCS method achieves higher rate-distortion performance with lower algorithm complexity as compared to standard QCS methods, conducive to delay-sensitive applications with large-scale signals.

Compressed sensing (CS) is an efficient technique to acquire sparse signals in many wireless applications to, e.g., reduce the amount of data and save low-power sensors batteries. This paper addresses efficient acquisition of sparse sources through quantized noisy compressive measurements where the encoder and decoder are realized by deep neural networks (DNNs). We devise a DNN based quantized compressed sensing (QCS) method aiming at minimizing the mean-square error of the signal reconstruction. Once trained offline, the proposed method enjoys extremely fast and low complexity decoding in the online communication phase. Simulation results demonstrate the superior rate-distortion performance of the proposed method compared to a polynomial-complexity QCS reconstruction scheme.

Sparsity is an attribute present in a myriad of natural signals and systems, occurring either inherently or after a suitable projection. Such signals with lots of zeros possess minimal degrees of freedom and are thus attractive from an implementation perspective in wireless networks. While sparsity has appeared for decades in various mathematical fields, the emergence of compressed sensing (CS) - the joint sampling and compression paradigm - in 2006 gave rise to plethora of novel communication designs that can efficiently exploit sparsity. In this monograph, we review several CS frameworks where sparsity is exploited to improve the quality of signal reconstruction/detection while reducing the use of radio and energy resources by decreasing, e.g., the sampling rate, transmission rate, and number of computations. The first part focuses on several advanced CS signal reconstruction techniques along with wireless applications. The second part deals with efficient data gathering and lossy compression techniques in wireless sensor networks. Finally, the third part addresses CS-driven designs for spectrum sensing and multi-user detection for cognitive and wireless communications.

In order to save energy of low-power sensors in Internet of Things applications, minimizing the number of bits to compress and communicate real-valued sources with a pre-defined distortion becomes crucial. In such a lossy source coding context, we study rate-distortion (RD) performance of various single-sensor quantized compressed sensing (QCS) schemes for compressing sparse signals via quantized/encoded noisy linear measurements. The paper combines and refines the recent advances of QCS algorithm designs and theoretical analysis. In particular, several practical symbol-by-symbol quantizer based QCS methods of different complexities relying on 1) compress-and-estimate, 2) estimate-and-compress, and 3) support-estimation-and-compress strategies are proposed. Simulation results demonstrate the RD performances of different schemes and compare them to the information-theoretic limits.

We study rate-distortion (RD) performance of various single-sensor compressed sensing (CS) schemes for acquiring sparse signals via quantized/encoded noisy linear measurements, motivated by low-power sensor applications. For such a quantized CS (QCS) context, the paper combines and refines our recent advances in algorithm designs and theoretical analysis. Practical symbol-by-symbol quantizer based QCS methods of different compression strategies are proposed. The compression limit of QCS - the remote RDF - is assessed through an analytical lower bound and a numerical approximation method. Simulation results compare the RD performances of different schemes.

Freshness of status update packets is essential for wide range of real-time Internet of things applications. In this paper, we study the information freshness of a single-server multi-source queueing model under a first-come first-served (FCFS) serving policy. In the considered model, each source independently generates status update packets according to a Poisson process. The information freshness of the status updates of each source is evaluated by the average age of information (AoI). We derive an approximate expression for the average AoI for a multi-source M/G/1 queueing model having a general service time distribution. Simulation results are provided to validate and assess the tightness of the proposed approximate expression for the average AoI in the M/G/1 queueing model where the service time follows a gamma distribution.

Information freshness is crucial in a wide range of wireless applications where a destination needs the most recent measurements of a remotely observed random process. In this paper, we study the information freshness of a single-server multi-source M/M/1 queueing model under a first-come first-served (FCFS) serving policy. The information freshness of the status updates of each source is evaluated by the average age of information (AoI). We derive an exact expression for the average AoI for the multi-source M/M/1 queueing model. Simulation results are provided to validate the derived exact expression for the average AoI.

We consider an M/M/1 status update system consisting of two independent sources and one server. We derive the average age of information (AoI) of each source using the stochastic hybrid systems (SHS) technique under the following packet management with self-preemptive serving policy. The system can contain at most two packets with different source indexes at the same time, i.e., one packet under service and one packet in the queue. When the system is empty, any arriving packet immediately enters the server. When the server is busy at an arrival of a packet, the possible packet of the same source in the system (either waiting in the queue or being served) is replaced by the fresh packet. Numerical results illustrate the effectiveness of the proposed packet management with self-preemptive serving policy compared to several baseline policies.

In this paper, we consider an M/M/1 status update system consisting of two independent sources, one server, and one sink. We consider the following last-come first-served (LCFS) prioritized packet management policy. When the system is empty, any arriving packet immediately enters the server; when the server is busy, a packet of a source waiting in the queue is replaced if a new packet of the same source arrives and the fresh packet goes at the head of the queue. We derive the average age of information (AoI) of the considered M/M/1 queueing model by using the stochastic hybrid systems (SHS) technique. Numerical results illustrate the effectiveness of the proposed packet management policy compared to several baseline policies.

We study the information freshness under three different source aware packet management policies in a status update system consisting of two independent sources and one server. The packets of each source are generated according to the Poisson process and the packets are served according to an exponentially distributed service time. We derive the average age of information (AoI) of each source using the stochastic hybrid systems (SHS) technique for each packet management policy. In Policy 1, the queue can contain at most two waiting packets at the same time (in addition to the packet under service), one packet of source 1 and one packet of source 2. When the server is busy at an arrival of a packet, the possible packet of the same source waiting in the queue (hence, source-aware) is replaced by the arrived fresh packet. In Policy 2, the system (i.e., the waiting queue and the server) can contain at most two packets, one from each source. When the server is busy at an arrival of a packet, the possible packet of the same source in the system is replaced by the fresh packet. Policy 3 is similar to Policy 2 but it does not permit preemption in service, i.e., while a packet is under service all new arrivals from the same source are blocked and cleared. Numerical results are provided to assess the fairness between sources and the sum average AoI of the proposed policies.

In the context of the next generation wireless networks, freshness of status update packets is essential for enabling the services where a destination needs the most recent measurements of various sensors. In this paper, we study the information freshness of a multi-source M/G/1 first-come first-served (FCFS) queueing model, where each source independently generates status update packets according to a Poisson process. The information freshness of the status updates of each source is evaluated using the average age of information (AoI). To this end, we derive a closed-form expression for the average AoI of each source. As particular cases of our general expressions, we also derive closed-form expressions of the average AoI for both multi-source M/M/1 and single-source M/G/1 queueing models.

We consider a multi-source status update system, where each source generates status update packets according to a Poisson process which are then served according to a generally distributed service time. For this multi-source M/G/1/1 queueing model, we consider a self-preemptive packet management policy and derive the moment generating functions (MGFs) of the age of information (AoI) and peak AoI of each source. According to the policy, an arriving fresh packet preempts the possible packet of the same source in the system. Furthermore, we derive the MGFs of the AoI and peak AoI for the globally preemptive and non-preemptive policies, for which only the average AoI and peak AoI have been derived earlier. Finally, we use the MGFs to derive the average AoI and peak AoI in a two-source M/G/1/1 queueing model under each policy. Numerical results show the effect of the service time distribution parameters on the average AoI. The results also highlight the importance of higher moments of the AoI.

We consider a status update system consisting of two independent sources and one server in which packets of each source are generated according to the Poisson process and packets are served according to an exponentially distributed service time. We derive the moment generating function (MGF) of the age of information (AoI) for each source in the system by using the stochastic hybrid systems (SHS) under two existing source-aware packet management policies which we term self-preemptive and non-preemptive policies. In the both policies, the system (i.e., the waiting queue and the server) can contain at most two packets, one packet of each source; when the server is busy and a new packet arrives, the possible packet of the same source in the waiting queue is replaced by the fresh packet. The main difference between the policies is that in the self-preemptive policy, the packet under service is replaced upon the arrival of a new packet from the same source, whereas in the non-preemptive policy, this new arriving packet is blocked and cleared. We use the derived MGF to find the first and second moments of the AoI and show the importance of higher moments.

We consider a multi-source status update system in which status updates are transmitted as packets containing the measured value of the monitored process and a time stamp representing the time when the sample was generated. The packets of each source are generated according to a Poisson process and served according to an exponentially distributed service time. We assume that the received status update packets need further processing before being used (hence, computation-intensive). This is mathematically modeled by an additional server at the sink. The sink server serves the packets according to an exponentially distributed service time. We introduce two packet management policies, a preemptive policy and a blocking policy, and derive the moment generating function (MGF) of the AoI of each source under the both policies. In the both policies, the system can contain at most two packets, one at the transmitter server and one at the sink server. In the preemptive policy, a new arriving packet preempts any possible packet that is currently under service regardless of the packets source index. In the blocking policy, when a server is busy at the arrival instant of a packet, the arriving packet is blocked and cleared. We assume that the same preemptive/blocking policy is employed in both the transmitter and sink server. Numerical results are provided to assess the results.

Freshness of status update packets is essential for enabling services where a destination needs the most recent measurements of various sensors. In this paper, we study the information freshness of single-server multi-source queueing models under a first-come first-served (FCFS) serving policy. In the considered model, each source independently generates status update packets according to a Poisson process. The information freshness of the status updates of each source is evaluated by the average age of information (AoI). We derive an exact expression for the average AoI for the case with exponentially distributed service time, i.e., for a multi-source M/M/1 queueing model. Moreover, we derive three approximate expressions for the average AoI for a multi-source M/G/1 queueing model having a general service time distribution. Simulation results are provided to validate the derived exact average AoI expression, to assess the tightness of the proposed approximations, and to demonstrate the AoI behavior for different system parameters.

Freshness of status update packets is essential for enabling a wide range of applications in wireless sensor networks (WSNs). Accordingly, we consider a WSN where sensors communicate status updates to a destination by contending for the channel access based on a carrier sense multiple access (CSMA) method. We analyze the worst case average age of information (AoI) and average peak AoI from the view of one sensor in a system where all the other sensors have a saturated queue. Numerical results illustrate the importance of optimizing the contention window size and the packet arrival rate to maximize the information freshness.

Freshness of status update packets is essential for enabling a wide range of Internet of Things (IoT) applications. In this paper, we consider a status update system in which various sensors are assigned to transmit status update packets of a physical process to a desired destination. We consider that the sensors share a wireless channel and contend for the channel access based on a carrier sense multiple access (CSMA) method. We study freshness of the status update system at the destination using the age of information (AoI) metric. To this end, we analyze the worst case average AoI for each sensor in the CSMA-based system. Numerical results show that the AoI in the CSMA-based system may dramatically increase when the number of sensors increases. Moreover, we observe that the contention window size and the packet arrival rate must be optimized since they have a critical role in the performance of the system.

We consider a status update system where multiple sensors communicate timely information about various random processes to a sink. The sensors share orthogonal sub-channels to transmit such information in the form of status update packets. A central controller can control the sampling actions of the sensors to trade-off between the transmit power consumption and information freshness which is quantified by the Age of Information (AoI). We jointly optimize the sampling action of each sensor, the transmit power allocation, and the sub-channel assignment to minimize the average total transmit power of all sensors, subject to a maximum average AoI constraint for each sensor. To solve the problem, we develop a dynamic control algorithm using the Lyapunov drift-plus-penalty method and provide optimality analysis of the algorithm. According to the Lyapunov drift-plus-penalty method, to solve the main problem, we need to solve an optimization problem in each time slot which is a mixed integer non-convex optimization problem. We propose a low-complexity sub-optimal solution for this per-slot optimization problem that provides near-optimal performance and we evaluate the computational complexity of the solution. Numerical results illustrate the performance of the proposed dynamic control algorithm and the performance of the sub-optimal solution for the per-slot optimization problem versus the different parameters of the system. The results show that the proposed dynamic control algorithm achieves more than 60 % saving in the average total transmit power compared to a baseline policy.

In this paper, we consider a system where multiple low-power sensors communicate timely information about a random process to a sink. The sensors share orthogonal subchannels to transmit such information in the form of status update packets. Freshness of the sensors information at the sink is characterized by the Age of Information (AoI), and the sensors can control the sampling policy by deciding whether to take a sample or not. We formulate an optimization problem to minimize the time average total transmit power of sensors by jointly optimizing the sampling action of each sensor, the transmit power allocation, and the subchannel assignment under the constraints on the maximum time average AoI and maximum power of each sensor. To solve the optimization problem, we use the Lyapunov drift-plus-penalty method. Numerical results show the performance of the proposed algorithm versus the different parameters of the system.

We study status updating under two-way delay in a system consisting of a sampler, a sink, and a controller residing at the sink. The controller drives the sampling process by sending request packets to the sampler. Upon receiving a request, the sampler generates a sample and transmits the status update packet to the sink. Transmissions of both request and status update packets encounter random delays. We develop optimal control policies to minimize the average age of information (AoI) using the tools of Markov decision processes in two scenarios. We begin with the system having at most one active request, i.e., a generated request for which the sink has not yet received a status update packet. Then, as the main distinctive feature of this paper, we initiate pipelined-type status updating by studying a system having at most two active requests. Furthermore, we conduct AoI analysis by deriving the average AoI expressions for the Zero-Wait-1, Zero-Wait-2, and Wait-1 policies. According to the Zero-Wait-1 policy, whenever a status update packet is delivered to the sink, a new request packet is inserted into the system. The Zero-Wait-2 policy operates similarly, except that the system can hold two active requests. According to the Wait-1 policy, whenever a status update packet is delivered to the sink, a new request is sent after a waiting time which is a function of the current AoI. Numerical results illustrate the performance of each status updating policy under varying system parameter values.

We present our vision for a departure from the established way of architecting and assessing communication networks, by incorporating the semantics of information, defined not necessarily as the meaning of the messages, but as their significance, possibly within a real-time constraint, relative to the purpose of the data exchange. We argue that research efforts must focus on laying the theoretical foundations of a redesign of the entire process of information generation, transmission, and usage for networked systems in unison by developing advanced semantic metrics for communications and control systems; an optimal sampling theory combining signal sparsity and timeliness, for real-time prediction/reconstruction/control under communication constraints and delays; temporally effective compressed sensing techniques for decision making and inference directly in the compressed domain; and semantic-aware data generation, channel coding, packetization, feedback, and multiple and random access schemes that reduce the volume of data and the energy consumption, increasing the number of supportable devices. This paradigm shift targets jointly optimal information gathering, information dissemination, and decision making policies in networked systems.

consider a status update system consisting of a sampler, a controller, a processing unit, a transmitter, and a sink. The sampler generates a sample upon receiving a request from the controller and the sample requires further processing before transmission, hence is computation-intensive. This is mathematically modeled by a server called process server. After processing the sample, the status update packet is generated and sent to the transmitter for delivery to the sink. This is mathematically modeled by a server called transmit server. The service time of each packet at the transmit and process servers follow geometric distributions. Moreover, we consider that the servers serve packets under the blocking policy, i.e., whenever a server is busy at the arrival time of a new packet, the new arriving packet is blocked and discarded. We analyze the average age of information (AoI) for two fixed policies, namely, 1) zero-wait-one policy and 2) zero-wait-blocking policy. According to the former policy, the controller requests sampling when there is no packet in the system. According to the zero-waitblocking policy, the controller requests a sample whenever the process server is idle. Furthermore, we develop an optimal control policy to minimize the average AoI using the tools of Markov decision process (MDP). In numerical results, we evaluate the performance of the policies under different system parameters. Moreover, we analyze the structure of the optimal policy.

We consider a multi-source hybrid automatic repeat request (HARQ) based system, where a transmitter sends status update packets of random arrival (i.e., uncontrollable sampling) and generate-at-will (i.e., controllable sampling) sources to a destination through an error-prone channel. We develop transmission scheduling policies to minimize the average number of transmissions subject to an average age of information (AoI) constraint. First, we consider known environment (i.e., known system statistics) and develop a near-optimal deterministic transmission policy and a low-complexity dynamic transmission (LC-DT) policy. The former policy is derived by casting the main problem into a constrained Markov decision process (CMDP) problem, which is then solved using the Lagrangian relaxation, relative value iteration algorithm, and bisection. The LC-DT policy is developed via the drift-plus-penalty (DPP) method by transforming the main problem into a sequence of per-slot problems. Finally, we consider unknown environment and devise a learning-based transmission policy by relaxing the CMDP problem into an MDP problem using the DPP method and then adopting the deep Q-learning algorithm. Numerical results show that the proposed policies achieve near-optimal performance and illustrate the benefits of HARQ in status updating.

We study the problem of remote tracking in an energy-harvesting enabled status update system consisting of an information source, a sampler, a transmitter, and a monitor. The information source is modeled as a finite-state Markov chain. The sampler samples the source, and the transmitter transmits the taken samples to the monitor. We consider both sampling and transmission costs, and thus, the source is not fully observable. The primary objective is to determine the optimal joint sampling and transmission policies based on a goal-oriented metric, defined by a generic distortion function. We first formulate a stochastic optimization problem and cast it into a partially observable Markov decision process (POMDP) problem. Subsequently, we employ the notion of belief state and characterize the belief space through the age of information (AoI) to convert the problem into a finite-state MDP problem, which is then solved via the relative value iteration algorithm. We also explore different estimation strategies at the monitor and examine their impact on the system performance. The simulation results show the effectiveness of the derived policy and reveal that, depending on the source dynamic, the choice of estimation strategy itself can significantly influence the overall performance.

We address the real-time tracking problem of a partially observable Markov source under sampling and transmission costs in an energy harvesting system with an unreliable communication channel. We provide a semantic-aware optimal sampling and transmission policy that minimizes the average value of a general distortion subject to an energy causality constraint. We formulate a partially observable Markov decision process (POMDP) problem. To solve the problem, we cast it into a belief MDP problem. Subsequently, by effectively bounding the belief space, we formulate a finite-state MDP problem, which is solved using relative value iteration. The simulation results demonstrate the effectiveness of the derived policy and highlight the significant impact of the source dynamics on performance.

We develop online scheduling policies to minimize the sum average age of information (AoI) subject to transmission capacity and long-run average resource constraints in a multi-source two-hop system, where independent sources randomly generate status update packets which are sent to the destination via a relay through error-prone links. A stochastic optimization problem is formulated and solved in known and unknown environments. For the known environment, an online nearoptimal low-complexity policy is developed using the drift-plus-penalty method. For the unknown environment, a deep reinforcement learning policy is developed by employing the Lyapunov optimization theory and a dueling double deep Q-network. Simulation results show up to 136% performance improvement of the proposed policy compared to a greedy-based baseline policy.

We consider a multi-source relaying system where independent sources randomly generate status update packets which are sent to the destination with the aid of a relay through unreliable links. We develop transmission scheduling policies to minimize the weighted sum average age of information (AoI) subject to transmission capacity and long-run average resource constraints. We formulate a stochastic control optimization problem and solve it using a constrained Markov decision process (CMDP) approach and a drift-plus-penalty method. The CMDP problem is solved by transforming it into an MDP problem using the Lagrangian relaxation method. We theoretically analyze the structure of optimal policies for the MDP problem and subsequently propose a structure-aware algorithm that returns a practical near-optimal policy. Using the drift-plus-penalty method, we devise a near-optimal low-complexity policy that performs the scheduling decisions dynamically. We also develop a model-free deep reinforcement learning policy for which the Lyapunov optimization theory and a dueling double deep Q-network are employed. The complexities of the proposed policies are analyzed. Simulation results are provided to assess the performance of our policies and validate the theoretical results. The results show up to 91% performance improvement compared to a baseline policy.

This letter provides query-age-optimal joint sampling and transmission scheduling policies for a heterogeneous status update system, consisting of a stochastic arrival and a generate-at-will source, with an unreliable channel. Our main goal is to minimize the average query age of information (QAoI) subject to average sampling, average transmission, and per-slot transmission constraints. To this end, an optimization problem is formulated and solved by casting it into a linear program. We also provide a low-complexity near-optimal policy using the notion of weakly-coupled constrained Markov decision processes. The numerical results show up to 32% performance improvement by the proposed policies compared with a benchmark policy.

We consider a multi-source relaying system where the sources independently and randomly generate status update packets which are sent to the destination with the aid of a buffer-aided relay through unreliable links. We formulate a stochastic optimization problem aiming to minimize the sum average age of information (AAoI) of sources under per-slot transmission capacity constraints and a long-run average resource constraint. To solve the problem, we recast it as a constrained Markov decision process (CMDP) problem and adopt the Lagrangian method. We analyze the structure of an optimal policy for the resulting MDP problem that possesses a switching-type structure. We propose an algorithm that obtains a stationary deterministic near-optimal policy, establishing a benchmark for the system. Simulation results show the effectiveness of our algorithm compared to benchmark algorithms.